deploy a database to the aws server

[gizmola]

docker and vagrant aren't doing the same job. Vagrant is really orchestration. It's a set of scripts and commands that can manage an environment, but it doesn't include virtualization technology itself, so it must use or interact with some other virtualization or cloud service. It doesn't include configuration management like chef, puppet or ansible, but it will work with them, running those systems to configure things. What it does is provide a way to package all these different technologies, so you can then duplicate them or pass them along to others. The idea is that anyone else running vagrant can take your vagrant files, and get the same environment you have. For that reason it's popular as a development tool for teams. As a team lead, I can package up our application development environment in a vagrant, and give it to you as a new employee and you are up and running in 20 minutes with no need to understand all the parts, configure things, load data, etc. It's not magic, but vagrant gives you plumbing to use other technologies that do everything else. So in that way, vagrant is capable of working with Docker.

 

There is also lots of overlap between these different techs, so Docker can do the same things that vagrant does by itself in many cases. It ends up being comfort level and what you know.

 

Docker is a container system that lets you start containers, which are typically self contained application units. For example, you would have a Docker container that just runs mysql and a separate one that just runs nginx. You can set up the containers however you want. It comes with cluster management tools (swarm) that let you work with nodes of computing instances and deploy work (start and stop containers).

 

There is a lot to all of this stuff, but what I realized from your explanation of what you are currently doing, is that your comfort level with unix is not great right now. The cool thing is that you can pop up a linux box with the distro of your choice and learn about it which in the old days might have taken quite a lot of work.

 

For example, I wrote some tutorials about setting up and running LAMP inside Virtualbox on a windows workstation back in 2008/2009:

 

http://www.gizmola.com/blog/archives/95-Run-a-Centos-Lamp-development-server-on-XP,-Vista-or-Win-7-using-VirtualBox.html

http://www.gizmola.com/blog/archives/blog/archives/97-Centos-Virtual-LAMP-server-Part-II.html

 

It might be interesting to look at those, and even do a linux install yourself. The only thing I wouldn't do would be use the loopback device technique, as you can just use host networking or a bridge with virtualbox.

 

A Bridge network is ideal, as virtualbox will create a private network shared between your workstation and the virtualbox machine(s). At that point, you can access the virtualbox server by it's allocated IP.

 

It's very important that you install and keep virtualbox guest additions current. One of the nice things about vagrant is that there is a plugin that does that work for you and saves a huge amount of time. The guest additions are what support shared folders. So that way, you put all your code you are working on in a structure in a shared folder. If you are using best practices this code is under the control of git.

 

Inside your VM, you have set that shared folder to be the directory structure your web server/php is serving up.

 

This allows you to use whatever IDE or editor you want under windows, and then see the changes you've made instantly, as hosted on your linux based virtual host.

 

What happens with vagrant is that someone did an installation similar to what I do in the tutorial, and then made an image out of it. Once registered as a base image, anyone can use that distro inside a vagrant.

 

If there are specific things that I need to be different, I can use chef recipes to change the stuff I need to be configured differently, or add to the software stack.

 

While Docker is amazing cutting edge tech, I would urge you to concentrate on learning more about hosting on Linux, and use vagrant for that. It will help you understand how to do things manually in an ec2 host, and once you understand how to do something manually you can look at how to automate.

[gizmola]

Does this imply that there need to be two server instances, one the application server and the other the database server? Or can the two subnets be created on the same single instance?

Yes you would need 2 instances. However you also have the option of starting up an RDS in the db subnet. My main point is that scalability is going to be focused on computation and memory initially.

 

In other words, your Apache/PHP are the application servers. You will have n "application servers" to every DB. DB's can be scaled up by upgrading to larger instance types.

 

In production people often make the mistake with mysql of putting all this on one server, which means that the application has to compete with resources with the DB. With mysql and InnoDB you want to allocate the majority of server memory to your innodb buffer pool.

 

With apache/php you want all the memory available for apache. As load increases, apache will require more connections, more child processes, and it will expand to address load. If your DB is on the same box, the competing needs of each component compete with each other.

 

Database clustering is complicated and in the relational world there is no free lunch. You have to start baking things into your infrastructure like replication, and read vs write, so it's not like you can just throw up 2 new mysql db's with a load balancer and get more capacity.

 

With apache/php that is exactly what you can do, and since your application logic exists there, it's the first place you are going to need to scale.

 

Of course one of the ways to increase performance and minimize db load is to use caching servers. Memcache and Redis are both highly popular choices. This is why I mentioned those, but you can start with a simple 2 tier approach and layer in caching later when you are ready for the addition of that extra complexity.

[ajoo]

HI Gizmola !! 

 

Wow ! That's a lot of valuable information there !! I'll assimilate it and revert ! I have skimmed through the post and I think that I am doing most of what you have mentioned in the first one regarding vagrant. 

 

Thanks to Guru Jacques and Benanamen I started working with Vagrant and VM's and understand quite some of their working. Enough to get me setting up a VM with vagrant. I'll try and co-relate what I am currently doing with your suggestions and see what I may be missing out. I am using git on my code for versioning as well. I'll go through your tutorials and would be back soon with any doubts.

 

Thanks a ton !

[ajoo]

Hi !

 

 

 

The only thing I wouldn't do would be use the loopback device technique

 

Can you please explain this. I know loopback refers to the use of 127.0.0.1 address. But are we referring to the loopback in the VM? I don't think I am doing this but how can I know and be sure that this is not what I am doing ? Here is the snapshot of the booting of my VM and can maybe give you a clue. As you can see it uses the NAT and the bridge adapter.

 

 

==> default: Attempting graceful shutdown of VM...

==> default: Checking if box 'box-cutter/ubuntu1404-i386' is up to date...

==> default: Clearing any previously set forwarded ports...

==> default: Clearing any previously set network interfaces...

==> default: Preparing network interfaces based on configuration...

    default: Adapter 1: nat

    default: Adapter 2: bridged

==> default: Forwarding ports...

    default: 22 (guest) => 2222 (host) (adapter 1)

==> default: Booting VM...

==> default: Waiting for machine to boot. This may take a few minutes...

    default: SSH address: 127.0.0.1:2222

    default: SSH username: vagrant

    default: SSH auth method: private key

  

 

The VM gets its own unique IP once the VM is ready and which I access using the ifconfig command, which is also what I use to set in the hosts file in windows to access my website from the host.

 

Most of what you said I am already doing with vagrant and managing / updating my code in while testing it using the VM.

 

Thanks for clarifying about the db instances not being  scalable which is what I thought them to be just like the EC2 server instances. 

Your replies were a great read. I have yet to check out your tutorials which I will shortly.

 

Thanks loads !!