The Delta file format supports ACID properties by storing data in parquet files and actions in JSON files. This storage format allows the developer to time travel to a given version number or timestamp. However, it means the files keep on growing until an optimize statement (file compaction) and vacuum statement (file removal) are executed. Once these commands are executed, the time travel is limited to the file retention in hours. The default file retention is 168 hours or 7 days.
If you are writing single threaded notebooks, the Delta file format looks very attractive. If you do not update