From raison at chatsubo.net Mon Dec 12 19:05:47 2016 From: raison at chatsubo.net (Kevin Raison) Date: Mon, 12 Dec 2016 11:05:47 -0800 Subject: Major performance improvements to VG Message-ID: Folks, I recently checked in a few breaking changes to the VG3 repo that you should be aware of. In particular, the hashing scheme used for vertices and edges was not distributing keys very well and graphs would slow down terribly as they grew very large (1,000,000+ nodes). I have updated the UUID generation code as well as the hash functions for vertices, edges, ve indices and vev indices. My performance tests show an improvement in loading a 10,000,000 node snapshot from 2.5 days to about 2 hours. Hash key distribution is largely responsible for this change, but so is an optimization made to the hashing function. As of commit 58f87d60e767d868cf30b8e6f1ec0bfc9d6d0b1e , existing graphs will not work. I suggest that you take a snapshot of your graph(s) and reload them using the REPLAY function. Please let me know if you have any questions or concerns. I am also curious how many active users VG has at this point; can folks chime in about their usage of the database? Cheers, Kevin Raison