Recently I had a severe problem with SQL endpoints and I was required to build a quick workaround for this.
The problem brought to light details we don’t know about SQL Endpoints and the care we need to have.
SQL Endpoint has a Cache
The SQL Endpoint needs to retrieve data from the storage, but it keeps its own cache. The SQL Endpoint cache is refreshed every 15 seconds.
This means that between a notebook write data in a lakehouse and the SQL Endpoint make this data available, there may be a 15 second delay.
SQL Endpoint is a