logo
podcast Podcast
get help Get Unstuck

Monitoring progress of a big database import

Without this, it feels like it takes forever

Joel Clermont
Joel Clermont
2024-03-18

I recently helped someone migrate a database from a MySQL instance running on their web server to a managed MySQL cluster hosted by Digital Ocean.

There are a few ways to do this, but since the database wasn't huge (under 20 GB), we decided for a simple mysqldump on the old server, followed by mysql import against the target server.

But if you just run mysql < some_big_dump.sql you have no indication how long it will take or how far along the process is. You just have to wait patiently until it's done.

To monitor the progress, you can use pv (pipe viewer). It's a small utility that shows the progress of data through a pipeline.

Here's how you can use it:

pv some_big_dump.sql | mysql

Notice we had to flip the order of the commands. pv reads from standard input and writes to standard output.

Best of all, this tool is not MySQL specific. You can use it with any command that reads from standard input and writes to standard output.

Here to help,

Joel

P.S. Would you like some help moving your database off your web server and onto something more reliable? Let's talk!

Toss a coin in the jar if you found this helpful.
Want a tip like this in your inbox every weekday? Sign up below 👇🏼
email
No spam. Only real-world advice you can use.