VistaDB 5.1 Released with Big Performance Improvements
We’ve published a new release of VistaDB with our biggest performance improvements ever! We set out a year ago to make a big improvement in the optimizer for VistaDB and ended up not just doing that but also dramatically improving the speed of the underlying storage engine!
You can download this latest version immediately, or read more about it below.
Lets See Those Performance Improvements!
Through the 5.1 development program we’ve been working with a special list of queries that were intended to highlight performance issues with the 5.0 engine – either where it was slower than 4.x (a few cases) or just where VistaDB has always been slower than the query would lead you to believe. Here’s the final results:
With every build we went to a specific system and ran this set of performance tests and checked our progress. Our goal was simple – 5.1 had to out-perform every prior release of VistaDB and offer real-world gains for our customers.
File Size and Performance
One area we made a tradeoff is in how VistaDB stores values like integers and dates. Historically it has used a compression technique that stored these values as deltas within a page – storing the difference between consecutive values instead of the actual value. This is much more space efficient, but requires parsing the whole page of data to read any value. We found dramatic performance gains by eliminating this to allow the engine to parse a single row on a page or even a single column in a row. We explored other options borrowed from Loupe to encode these values but the net effect was a 5-10% reduction in performance across the board for a 15% reduction in file size.
In the end, we feel the performance cost was too high. In our tests, a VistaDB 5.1 database is about 20% larger than a 5.0 database. For the dramatic gains in performance it’s a trade well worth making!
Upgrading to This Release
To implement the storage engine optimizations listed above it was necessary to change the underlying storage format used to serialize some data types. To support this version 5.1 has a new data format. As always, to upgrade to the latest data format just pack the database. Once this is complete it will not be readable by an older version of the VistaDB engine including 5.0. VistaDB 5.1 can read and write any version 5.0 database and read any VistaDB 3 or later database. Until the database is updated to the latest format the new storage optimizations won’t be available.
Because the file format has changed and the VistaDB engine can be installed in the Global Assembly Cache (GAC) the assembly version has been changed. This ensures that an application built against VistaDB 5.0 won’t accidentally start using the VistaDB 5.1 engine and potentially upgrade its database to a format that the engine it shipped with can’t read. While the number of scenarios where this type of problem could occur are few, the consequences would be confusing and difficult to troubleshoot so the decision was made to prevent them.
To accommodate this assembly version change the assembly versions of the VistaDB Entity Framework providers and other assemblies that reference the engine have been updated as well. The good news is that most configurations don’t include the assembly version and are unaffected by this change. For more information, consult the release notes.