You can ask MySQL/MariaDB how much storage each table uses with a query. Comparing this to what the metrics show, there's a difference in disk usage.
First. Let's be clear, metrics is always right. It looks at the actual file system. Any query you can run gives you an idea on where the storage is, but it will most likely show less storage than is actually being used on disk. System log files, binary logs, left over files... all add up and they are not all accounted for by an sql query.
You can use the upsun db:size cli command to get database size, but these days it's probably better to use upsun disk to look at the disk. If you do need to use a query to look at the database table size, some form of the following query can be used:
MariaDB [(none)]>
SELECT
table_name,
round((data_length+index_length)/1048576,2) as size_MB,
round((data_free)/1048576,2) as allocated_but_unused_MB
FROM information_schema.tables
WHERE data_free > 1048576*10
ORDER BY data_free DESC;
+-------------------------------+---------+-------------------------+
| table_name | size_MB | allocated_but_unused_MB |
+-------------------------------+---------+-------------------------+
| magento_logging_event_changes | 1.03 | 109.00 |
| quote_item | 113.20 | 93.00 |
| quote_item_option | 61.09 | 59.00 |
| avatax_log | 101.98 | 14.00 |
| importexport_importdata | 0.03 | 11.00 |
| quote_address | 18.33 | 11.00 |
+-------------------------------+---------+-------------------------+What is often forgotten is the data_free column. data_free is the space that is not used anymore (rows that were DELETEed). But that InnoDB has not reclaimed. This happens because InnoDB cannot shrink the table datafile on disk if rows are removed in the middle of the data file. There's a very good article on the subject here: https://devcenter.upsun.com/posts/the-mystery-of-the-shrinking-database-dump/
You'd see cache tables, log tables, etc... will have a lot of data_free space due to them doing a lot of INSERTs followed by DELETEs.
You can reclaim it by:
- TRUNCATEing the table (destructive, will remove data!), and that's very quick if you don't value the data in the table (ie:
watchdogtable ). - OPTIMIZE TABLE (which won't always work because InnoDB gets to decide if it needs to do a recreate or not)
- running
upsun db:size --cleanupcli command. Which will force a recreate of the table non-destructively. It will copy all existing data to a new data file.
However, even when you don't have any data_free there will still be storage unaccounted for.
One common issue is Full text indexes, since they are stored in a separate data file and them growing in size goes unnoticed. So be careful which index you create.
root@mysql.0:/mnt/data/mysql/main# ls -lSh FTS*| head -10
-rw-rw---- 1 app app 804M Oct 29 09:22 FTS_0000000000009fe3_000000000001af88_INDEX_1.ibd
-rw-rw---- 1 app app 472M Oct 29 03:28 FTS_0000000000009fe3_DELETED.ibd
-rw-rw---- 1 app app 456M Oct 29 09:22 FTS_0000000000009fe3_000000000001af88_INDEX_2.ibd
-rw-rw---- 1 app app 356M Oct 29 09:22 FTS_0000000000009fe3_000000000001af88_INDEX_5.ibd
-rw-rw---- 1 app app 316M Oct 29 09:22 FTS_0000000000009fe3_000000000001af88_INDEX_4.ibd
-rw-rw---- 1 app app 268M Oct 29 09:22 FTS_0000000000009fe3_000000000001af88_INDEX_3.ibd
-rw-rw---- 1 app app 144M Oct 29 09:22 FTS_0000000000009fe3_000000000001af88_INDEX_6.ibd
-rw-rw---- 1 app app 23M Oct 29 09:22 FTS_0000000000004a5f_000000000000e5d3_INDEX_1.ibd
-rw-rw---- 1 app app 20M Oct 29 09:22 FTS_0000000000000a1b_0000000000001f1d_INDEX_1.ibd
-rw-rw---- 1 app app 17M Oct 29 09:22 FTS_0000000000000a33_0000000000001f50_INDEX_1.ibd
Secondly, if you have activated a mariadb-replica, the primary database node will have binary logging turned on. Depending on how much data is written per hour, binary logs can grow to a substantial size. Do increase storage ahead of time if you are going to use a replica.
You'll never get full parity between on disk storage reports and database query disk usage, but it should be relatively close if you account for the above. Should there be a very large discrepancy feel free to make a support ticket so we can have an in depth look at the container.
Comments
Please sign in to leave a comment.