Because XTDB uses a transaction log and document store, it's possible to purge a document from the document store so that the only thing that remains is an irreversibly hashed document id on the transaction log. This isn't crufty. Using a reference like this is a very common, good solution to GDPR compliance.
A system that's truly immutable in all parts is elegant, but unfortunately the real world is not as easily modelled. Being able to truly expunge data is actually needed at times. And it's not just GDPR. What happens when something truly sensitive is written to an immutable store?
I'd like to be able to know my data was purged, but still...
It's a very good article but I'm sure that the writer would agree that whether you want or need to remember history or not depends on the use case. Philosophically it is often better to forget. It's a way of dealing with change. Forgive and forget. On the other hand "those who dont' understand history" and so on. Also a lot of the time we really don't care about the previous states of something because those states are of no use to us now. We just care about what we can do with it now. We cannot go back to the past to use the thing so the past state of something is useless to us. This starts to break down in a virtual world where the past state of rest of the world could also be stored and so you could make some use of the past thing in that past virtual world. But most useful virtual systems are linked to things in the real world which we can only use in the present.
I went off on a tangent and found out about the EVICT transaction type in XTDB, which removes data from the logs which are otherwise immutable.
For some reason this GDPR compliant feature just makes me angry. It's so... so.... crufty!
Is it just me? I'd really like to know why this angers me so much? I'd like to be able to know my data was purged, but still.... arrggghhhh
Because XTDB uses a transaction log and document store, it's possible to purge a document from the document store so that the only thing that remains is an irreversibly hashed document id on the transaction log. This isn't crufty. Using a reference like this is a very common, good solution to GDPR compliance.
A system that's truly immutable in all parts is elegant, but unfortunately the real world is not as easily modelled. Being able to truly expunge data is actually needed at times. And it's not just GDPR. What happens when something truly sensitive is written to an immutable store?
So do you want the feature or not?! :)
It's a very good article but I'm sure that the writer would agree that whether you want or need to remember history or not depends on the use case. Philosophically it is often better to forget. It's a way of dealing with change. Forgive and forget. On the other hand "those who dont' understand history" and so on. Also a lot of the time we really don't care about the previous states of something because those states are of no use to us now. We just care about what we can do with it now. We cannot go back to the past to use the thing so the past state of something is useless to us. This starts to break down in a virtual world where the past state of rest of the world could also be stored and so you could make some use of the past thing in that past virtual world. But most useful virtual systems are linked to things in the real world which we can only use in the present.