Upgrade into FreeBSD Bhyve was successfull

Published Date: 2021/07/25 by: DaVieS

As I told before we were started to migrate our infrastructure from Linux/KVM (FreeBSD) into FreeBSD/Bhyve (FreeBSD).

I announce that we succesfully finished the task with minor downtime.
It was affected all of our servers world-wide.

Meantime we performed querterly security updates and security audit, no problems found so far.
The "Ghost" module which is our SEO backend also updated and we added a lot more resources that allows it to process sites faster.

However we are proud that we are using our new (BVCP) technology to manage Bhyve virtual machines, but the FreeBSD Bhyve itself might worth a little explanation how is it going now ... what are the differences about speed and stability.

Stability

FreeBSD OS is very stable out-of-the-box, but you are still allowed to screw up things, like we done before when we removed a bridge interface with network interfaces attached to it. That leads to kernel panic, so FreeBSD is very permissive for system administrators, so you should know what you doing is absolutely standing at this point.

The Bhyve itself is just working, it uses a very minimal overhead so you have a significantly more resources left compared to KVM with the cost of memory ballooning. However memory ballooning caused some trouble before and I wasn't statisfied at all.

Stress is the Bhyve's middle name, because when we moved our ton of VMs into Bhyve while we running production services already in the background the copy-source "KVM" is just hang-up with ZFS on linux many times but not the FreeBSD with Bhyve, I let you know we maxed out the I/O on both servers.

Speed

Very hard to explain, but Bhyve is seems little faster than KVM especially on I/O actions.

I don't know why (yet) but network operations are done faster and reliable with Bhyve and trust me I just mention our sysadmin ToolBox software that actually making at least a million connections per minute and was not talked about the mainline SPAM filter that managing messages accross datacenters already. Packet loss is not typical at all.

Under KVM we experienced some network issues on large packet numbers, we had to tune the linux sysctl's and the VM too. Same with Read/Write and random I/O on HDD. So meanwhile the linux usually hang up on large datasets pending to write out, the FreeBSD manage much more better, only micro-laggs occurs.

Conclusion

And the reason why Im forcing FreeBSD:

Back in time FreeBSD was hell risky OS because it was very challenging to get a fully supported hardware.
But now linux is going on a trend that I simply don't like. Strange kernel messages, errors, ZoL self corruption, etc.
In the past 2-3 years I experienced too much errors from Linux Distros: ("Centos, Debian, Suse, Fedora, etc..") meanwhile much less on FreeBSD.

And with eye of sysadmin: while we are running things that serving a lot companies, and ppl. depending on our services too, we should guarantee that all the services will be available. For me? - does matter how I can sleep at nights.

But now there is great chance that FreeBSD win all those people who just do not like where the linux and GPL stuffs heading into, I was there in some case when the Windows replaced into linux as industrial-scale even there was not 'official support', same happening now with FreeBSD. I see more and more FreeBSD every day.

I can believe that FreeBSD will not stop here and get more popular meanwhile it reserves the tradition. For example when I last checked Bhyve some years ago it was completely a mess and now we are running from it! Amazing!

So switching to FreeBSD and FreeBSD Bhyve absolutely worth the effort for me!