Published Date: 2021/11/26 by: DaVieS
I could say I saw a everthing but not, here is the case:
There is a VM Host with ZFS Storage, direct attached, and there is a Linux VPS with ext4 filesystem, running mySQL Database.
Without sign of error, suddenly the VPS kernel log started spamming:
[23564.244561] blk_update_request: I/O error, dev vda, sector 960227912
[23564.375360] blk_update_request: I/O error, dev vda, sector 960227912
and things went south... The database server has corrupted and fails to start, crash or hung up immediately.
So recovering the mySQL's InnoDB (.ibd) files was very challenging, there was no luck with dd, ddrescue, so wrote a software that could help.
So the mistery,
- Physical HDDs are OK
- ZFS with checksuming: No Error
- qemu-img check: OK
- fsck on the Guest: OK
- mounting .qcow2 file on the VM Host, OK bad drive, sector error.
This is the first time that I saw a sector error on a virtual HDD with excellent underlaying storage.
My guess is the ext4 corrupted and the kernel showing a misleading error message after NULL read.
This is how you can restore innoDB or any file from corrupted Disk.
That was an insane s@&k however a good thing in a bad story, we got richier by another software for data recovery.
If you like the article then don't forget to share!
BVCP 2.0.x Released
Our one of the most popular FREE-TO-USE Software BVCP reached the next milestone.Big thanks to the active community, a ton of feedbacks received..
We have been busy at 2023. Im personally very happy to see a lot of new technologies especially EV related researches.We continued to support our..
FreeBSD 14 fresh install breaks mc (Midnight Commander) subshell support
We are very happy to announce that - if you did not know already - one of the best Operating System released a new version. FreeBSD 14. This..
ZFS Replication Software
While we are working on BVCP 2.0 just another software made to handle ZFS Replications over multiple nodes. The software yet not released..
Bhyve UEFI drops into EFI shell, Linux wont boot Easy Workaround
EFI works that way that the installer places a file into a FAT32 partition in a regular directory as called "BOOT". The usual full path of..
Nexus Datalogger 2023
We're proudly present that we released out our next version of datalogger software as commerical product. Our first version of Datalogger..
Big Concern: Facebook hack, Google Chrome V8 Hack, Meta Support
Many users experiences that their account leaked through Facebook Advertisement platform.The BOT change their names to Kevin Konkrete Davis and..
We are doing some regular maintenance at 22:00 - 02:00 UTC. Some services might be distrupted at this time. ..
FreeBSD Bhyve Webadmin | BVCP 1.9.8 Just Released
BVCP 1.9.8 features New protocol to transmit VNC/RFB data as encrypted and compressed.Better, faster and more responsive UI.Colored..