Is the server "remote"? When you did your "command line mysql" verification, did you do that on the server or from your workstation?
A "decode" of the files generated by the mysql_ssl_rsa_setup:
ca.pem - the "certificate authority" public key for the self-signed certs that were generated. (You will want this file on your client)
client-key.pem - the "client" private key they generated. (You will want this file on your client)
client-cert.pem - the "client" certificate that was generated which goes along with the client-key.
These are the files you would need available to your client. Ideally you want to make copies of those files in a directory (not under the web root) of the server running your php application, with read only permissions, but still readable by the user that the php process is running as. You need to pass an actual or relative path to the files, when you make mysqli_ssl_set initialization.
mysqli_ssl_set($con,"/path/to/client-key.pem","/path/to/client-cert.pem","/path/to/ca.pem",NULL, NULL);