The MariaDB Table Crash Thing

[bmorrison]

I've been getting a lot of calls from panicked Vicidial admins because their dialers are breaking. The savvier ones tell me their "database keeps crashing and I don't know why?"

While this MariaDB bug has been great for my business in the short term, the frustration I hear in the voices of people who thought they were doing the right thing by installing using the Vicibox Server ISO leads me to suspect that the current situation is bad for the Vicidial ecosystem long term.

I understand completely that this is an upstream package problem and not the fault of Vicibox, but I humbly suggest that MariaDB gets dumped in favor of something that isn't making people confused/angry/etc…

[Kumba]

I'm all ears for a solution if anyone has any suggestions. The TMM bug looks like it was introduced somewhere in 5.5/5.6 and followed MySQL/MariaDB up through v.10.

I do hear some chatter about it being "fixed" in MariaDB v.10.1.6, but I have no definitive way of confirming that. I'll have to see if I can get 10.1.6 to compile and then try to cause the condition that generates the TMM files.

[bmorrison]

Looks like in November of 2015, Daniel Bartholomew expected to have repos of 10.1.10 up for LEAP:

Yes, a new repo, and a new set of LEAP VMs in buildbot. I'll get the
VMs set up so we can start building and testing. We're too late for
10.1.9 (it is being released today and the downloads just went live, I
just haven't sent the announce email yet...) but 10.1.10 should have
working packages and a corresponding repository for LEAP.

Source:
https://lists.launchpad.net/maria-discuss/msg03129.html

I'll ping him and see what he has to say.

[Kumba]

The issue I am having is I ran into this on only 1 of the 28 databases what we maintain for ViciHost. The issue went away when I did the "fix" I posted about in the ViciBox v.7.0 thread. I have had clients say it reoccurs for them frequently. It's been over a month since I've ran into it on our stuff.

The good news is that there IS a version of MariaDB 10.1.6 for OpenSuSE. If those of you who are having this issue repeatedly want to install it and give feedback that would be extremely helpful. It's hard for me to test an issue that doesn't reoccur for us.

You will need to add an OBS repository to ViciBox in order to install the new version. It's an "official" repository so it should be relatively stable compared to an individual user's home repository. It might also be prudent to perform a full database back-up in case you run into any issues. How you backup a database is not covered here as that's a separate action.


Instructions for installing MariaDB 10.1.6+.

1) zypper ar http://download.opensuse.org/repositori ... abase.repo
2) zypper refresh (When prompted about the added repository, press 'a' and hit enter to always trust this repo)
3) service mysql stop
4) zypper dup
5) Delete any TMM files that you can find in the mysql directories in order to give it a good fresh start
7) service mysql start


If I get good feedback then I will add this repo to ViciBox 7 and release a new version.

[bmorrison]

I have personally seen this recur, I'm sad to report. Also these calls are (obviously) coming to me from affected users, those that are humming along without issue I'm not going to hear from, so I'll easily accept that these troubled servers are in a minority of the overall Vicibox v7 universe.

It appears MariaDB's buildbot is generating packages for OpenSUSE up through the 10.2 branch:

https://buildbot.askmonty.org/buildbot/

[Kumba]

It appears MariaDB's buildbot is generating packages for OpenSUSE up through the 10.2 branch:

https://buildbot.askmonty.org/buildbot/

I prefer to use official OpenSuSE OBS repos as the general build quality is higher and usually no complications during the install. A good chunk of the external build repos use a generic RPM style that usually isn't tailored for OpenSuSE but a "write once install everywhere, hopefully" approach. If you want to try those repos and report back with your findings that would be good. I would suggest trying the official OpenSuSE repository I posted first, and then if the issue is still there attempting the external packages.

[bmorrison]

I had to issue:

Code: Select all

zypper install mariadb-client-10.1.6-7.6.x86_64

Because of the vendor change, FYI.

[Kumba]

Corrected the instructions, it should have been zypper dup.

[Kumba]

So has anyone applied the update in production? any issues?

Feedback would be good.

[bmorrison]

I haven't had a problem yet, but the databases in question have been in production with 10.1.6 since only yesterday.

[Kumba]

I haven't had a problem yet, but the databases in question have been in production with 10.1.6 since only yesterday.

At least that's one person who is trying it in production. If you could please let me know in a week if you've had no issue or if it still occurs that would be great. Hopefully that takes care of the issue.

[Kumba]

So anyone had any issues since performing the MariaDB update?

[bmorrison]

No issues.

[Kumba]

Well, not much of a feedback pool, but I'm going to release ViciBox v.7.0.2 with MariaDB 10.1.6. I'll also update the bugfix thread about it. Thanks for the feedback.

[bmorrison]

That's fantastic news. Thanks for addressing this issue so quickly!

[davesdatasystems]

First week after the patch no issues.

However i had one small issue on getting everything working after the patch but a quick database optimization seemed to fix it.