I'm super sad to announce that in cause of bad server configuration, entire production database of prismo.news has been accidentally wiped out.
Without option to restore it. You can't imagine how angry i am right now but i assure you i'm gonna write an extended post about that incident with description of everything that happened and lessons learned.
I'm super sorry for your loss, i hope fediverse will forgive me one day.
Beside good resolutions (yes, I should do more backups) this made me wonder if (and how) a federation such as the Fediverse could be more robust against this type of failure.
In the Fediverse where content is duplicated by design it doesn't seem so difficult to recover pieces that are stored somewhere after a failure.
Of course, the devil is in the detail and there would be a number of such details to fix:
- can you trust the federation when it would give you pieces back if activities are not signed ?
- what about stuff which aren't activities (such as the list of account I am following) ? should client applications be involved in keeping copies of these informations ?
However I think that a federation should be more than the sum of its members and that seems to be a nice step forward.
The next step being to consider how the federation could preserve content when a node disappears permanently...