DBA fledgeling.

[certehfiable]

Ok I've read Fenway's stickied posts. That has to count for something, right?

 

Let me try to break this down into a concise neat package for everyone and thanks in advance for the help and experience.

 

CURRENT CONFIG:

CentOS 5.x running in 64 bit architecture.

MYSQL - 5.0.86-log compiled by Jason at utterramblings.

 

QUESTIONS and ISSUES: (what I need help with)

Diagnosing MYSQL queries. How to? Where to start? Sufficient log level? Tricks of the trade.

 

DETAILS:

I'm running an IPB (Invision Power Board) release 3.0.x and trying to optimize MYSQL and the queries associated with supporting that software. My attempts at asking IPB to look into this have failed. I take full responsibility as I should know more about what is actually going on instead of just believing a boxed product is designed to work for every environment. Hence why I'm here.

 

Now that we know what I'm trying to fix and/or diagnose possibly I can be more educated about the problem and solutions with some help from here.

 

MY.CNF

[client]
port=3306
socket=/var/lib/mysql/mysql.sock

[mysqld]
collation_server=utf8_unicode_ci
character_set_server=utf8
user=mysql
set_variable=local_infile=0
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

max_connections         =       200
key_buffer_size         =       256M
table_cache             =       3072
max_allowed_packet      =       16M
join_buffer_size        =       2M
sort_buffer_size       =        1M
read_buffer_size       =        1M
read_rnd_buffer_size   =        1M
max_heap_table_size     =       256M
tmp_table_size          =       256M
bulk_insert_buffer_size =       64M
myisam_sort_buffer_size=        128M
myisam_max_sort_file_size =     2G
myisam_max_extra_sort_file_size = 2G
myisam_repair_threads   =       4
myisam_recover
thread_cache_size       =       80
query_cache_type        =       1
query_cache_size        =       128M
query_cache_limit       =       8M
thread_concurrency      =       8

set-variable=innodb_buffer_pool_size=40M
set-variable=innodb_log_file_size=40M
set-variable=innodb_log_buffer_size=2M
set-variable=innodb_additional_mem_pool_size=2M
set-variable=innodb_max_dirty_pages_pct=90

skip_character_set_client_handshake
skip_federated
skip_locking
skip_bdb

log_slow_queries
log_long_format
memlock

server-id=1

[innodb]
innodb_data_home_dir = /var/lib/mysql/
innodb_data_file_path = ibdata1:100M:autoextend;ibdata2:10M:autoextend
innodb_log_arch_dir = /var/lib/mysql/
innodb_buffer_pool_size=40M
innodb_additional_mem_pool_size=2M
innodb_log_file_size = 40M
innodb_log_buffer_size = 2M
innodb_max_dirty_pages_pct = 90
innodb_flush_log_at_trx_commit = 1
innodb_flush_method=O_DSYNC
innodb_lock_wait_timeout = 50

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash

[isamchk]
key_buffer = 512M
sort_buffer_size = 512M
read_buffer = 8M
write_buffer = 8M

[myisamchk]
key_buffer = 512M
sort_buffer_size = 512M
read_buffer = 8M
write_buffer = 8M

[mysqlhotcopy]
interactive-timeout

[mysqld_safe]
open-files-limit = 8192

 

MYSQLTUNER REPORT

 >>  MySQLTuner 1.0.1 - Major Hayden <major@mhtx.net>
>>  Bug reports, feature requests, and downloads at http://mysqltuner.com/
>>  Run with '--help' for additional options and output filtering

-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.0.86-log
[OK] Operating on 64-bit architecture

-------- Storage Engine Statistics -------------------------------------------
[--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 1G (Tables: 377)
[--] Data in InnoDB tables: 11M (Tables: 144)
[!!] Total fragmented tables: 24

-------- Performance Metrics -------------------------------------------------
[--] Up for: 22h 25m 56s (6M q [82.139 qps], 269K conn, TX: 347B, RX: 36B)
[--] Reads / Writes: 36% / 64%
[--] Total buffers: 684.0M global + 5.2M per thread (200 max threads)
[OK] Maximum possible memory usage: 1.7G (21% of installed RAM)
[OK] Slow queries: 0% (190/6M)
[OK] Highest usage of available connections: 48% (97/200)
[OK] Key buffer size / total MyISAM indexes: 256.0M/220.9M
[OK] Key buffer hit rate: 100.0% (3B cached / 98K reads)
[OK] Query cache efficiency: 64.6% (2M cached / 3M selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 0% (2K temp sorts / 244K sorts)
[!!] Joins performed without indexes: 1394
[!!] Temporary tables created on disk: 31% (129K on disk / 404K total)
[OK] Thread cache hit rate: 99% (111 created / 269K connections)
[OK] Table cache hit rate: 39% (1K open / 4K opened)
[OK] Open file limit used: 23% (1K/8K)
[!!] Table locks acquired immediately: 78%
[OK] InnoDB data size / buffer pool: 11.1M/40.0M

-------- Recommendations -----------------------------------------------------
General recommendations:
    Run OPTIMIZE TABLE to defragment tables for better performance
    Adjust your join queries to always utilize indexes
    Temporary table size is already large - reduce result set size
    Reduce your SELECT DISTINCT queries without LIMIT clauses
    Optimize queries and/or use InnoDB to reduce lock wait
Variables to adjust:
    join_buffer_size (> 2.0M, or always use indexes with joins)

 

WHAT IS CONCERNING ME:

 

The fragmented tables keeps growing. I almost have to optimize them daily.

The joins with apparently no indexes.

The tables wait lock.

The fact that mysqltuner wants a bigger join_buffer than >2MB

 

 

Thoughts and where to start?

 

 

 

 

[fenway]

Thanks for following the posting guidelines -- good job, and welcome!

 

Since this is a commercial product, I find it surprising that the IPB team doesn't respond to your inquiries.

 

I agree that the "joins without indexes" is a problem -- you should turn on the slow query log, and in particular, see this regarding index usage.

 

As for temporary tables, that can simply be the result of ORDER / GROUP by clauses that don't use covering indexes -- quite common indeed.

 

As for fragmentation, how often are you DELETE-ing records?