website development cycle

[dardub]

I was wondering if anyone had any suggested resources (books/web pages) for setting up a proper testing server, how to push to a live site, version control and other web development practices.  I've been reading around about this svn, but nothing very concise for someone who has no experience with this.

 

Basically I'm looking for a guide on the best practices to develop a website or web application from start to finish. Let me know if you need more clarification. I understand that everyone has different methodologies, but I need something to start out with.

[dardub]

Hmm.. is this a crazy request?  Is there nothing good out there? 

[Mchl]

I guess there is no such guide. People/teams/companies are constantly trying new ideas in this field because whoever gets better streamlined development process, gains advantage over competition.

 

Specific tools that can be used also depend on what technologies are used for given project.

 

This subject is really broad...

[dardub]

I guess I can ask a more specific question.  My first hope was there would be a book out there to get me going. I have been doing some research on the matter.  The problem is this is my first job doing php programming and there's nobody at my work to ask questions like this to.  There isn't a good testing environment here and there's no version control either.

 

Right now our production and testing site are hosted remotely on linux servers.  The files I work on are on a local windows server, then I upload the files remotely if I want to view the changes.  Then the files are uploaded onto the productions server. Doesn't seem like a good system right? There is only one other person working on files doing front end stuff. 

 

I want to set up a local testing server where I work on the files and test the changes.  This local testing server would have an svn server as well.  Then I would upload the files to the staging area (former testing server).  A final check of the site would be done there.  Then we'd push to the production server?  Hosting provider said he can use rsync to do this.

 

Is this a proper method for website production? Is there anything else that I am missing.  Should there be a separate svn repository for the staging and production server as well.  And should all the svn repositories be served from the local testing server?

 

BTW, The procedure I'm talking about is for an existing site.  I will be working on a new site in the near future as well.

[Mchl]

Anything that formalises the development process is an improvement over having no rules at all.

 

For setting up SVN repository, I'll refer you to http://svnbook.red-bean.com/ which explains the usual method pretty well.

 

If the website is not very large, you can install development server on your own machine to check your changes locally for any obvious bugs before committing them to repository's trunk.

Then you could set up a testing server, where you would checkout current revision of trunk, and run a set of automated and/or manual tests designed to test required functionality, and also for bugs that were present in previous versions (if they appear again, it's called a regression).

 

Once you decided that selected revision will be deployed to a live site, you create a tag out of it in the repository (see book above).

Your staging server should ideally have exactly same configuration like your production (live) server. You can checkout a tag created in previous test, load database with data from live website, and again check if everything is all right.

 

Once this step has been cleared, you should be ready to update your production server with new version.

[dardub]

Thanks for your help Mchl.  I think I understood about half of what you said.    I'll have to start reading the book to get a good understanding of svn.

 

From what I gather, the entire process is using the same svn server and repository?  And I assume it's fine to have a local svn server?  I just found out that we have a local linux server that never got finished setting up. I'll have to get that going.

 

I'll probably have more questions after I finish the book. Thanks.

[Mchl]

Yes, you use one version control server along the way (and also be aware SVN is not the only version control software available, some people think git or bazaar are better)

[jadamski]

http://en.wikipedia.org/wiki/Software_development_process

 

I don't know if this will help you specifically but here's how I would approach the problem.

What kind of testing environment, implementation plan etc. really depends on you and the model or style of development you work under most of the time. Are you a single developer, or are you on a team? Do you have to report to anyone or do you develop for yourself.

 

A good web application usually has three parts the UI, Platform and Database; so are you testing all three or just one part? What I recommend is to spend time on the WIKI's until you see yourself reflected in the advice there. Once you see your style and the tools you use being mentioned, latch on to that topic as a possible work method for you.

 

Start with the general link above which about the entire process. Investigate what appears in the right hand column on most of the wiki pages. Those are links to the more specific subject matter. Make a note when a reference or process is like the situation you are working in. You didn't mention your tools etc, so I am assuming the LAMP stack. SVC is only one player in the Version Control universe so look at some others for simplicity.

 

If you are an Eclipse user, you should tag what ever is Eclipse related until you can see a pattern emerging (likewise for Netbeans, Visual Studio, whatever).

 

I teach this stuff and its what I try to get my students to do. Development is not done in a vacuum and every thing about it depends on you and the way in which you do your work. Try the wikis from the top and I think you will start to see what I mean.

 

 

[dardub]

Let me know if I should start a new thread.  I was working on other things, but I'm slowly working out what to do about svn today.

 

So right now we have a remote production server and staging server.  We will soon have a local linux testing server up and running.  My thoughts were to have the svnserver and host the repository on the local testing server.

 

I have some more specific questions now. Would there be a problem with developing on the working copy on the local testing server? I'm working with Magento, and it does not work too nicely with windows, from what I'm experiencing.  And I'd rather not make us switch to linux on our desktop machines (partly because we use adobe cs). At the moment there are only two of us making modifications to the site.

 

I suppose I'm still a little cloudy on some aspects. Is the following something that could work?

 

1. Testing is a working copy of the trunk in the www dir on the local testing server.

 

2. Manual testing is done and necessary changes are made.

 

3. Create a tag and name the folder accordingly (release-1.1.0 or whatever)

 

4. Checkout the current tag into the staging server public directory, import newest db. 

 

5. Perform manual testing.

 

6. After testing has passed, use rsync to update production server, excluding .svn directories, cache, etc..

 

Anyone see a problem with having multiple working copies in the www testing directory.  One working copy for each individual (2 of us ATM)?

 

Sorry if my explanation doesn't make sense. I was thinking out loud when I wrote this, and was making changes along they way.

 

 

 

 

[roopurt18]

IMO do not host your SVN repos on the testing server.  A test server is just that!  Anyone coming into contact with a test server should feel free to assume nothing on the test server is critical.  Therefore anything on a test server can be deleted, reinstalled, replaced, etc.

 

I recommend you create your repos on a stable server within your company or organization.  The target server should have a firewall, disaster recovery plan, etc.

[dardub]

The other local server we have is a windows domain server with a local and offsite backup.  Would you recommend using the windows server with virtual svn server? Or putting svn server on our staging server?

[Mchl]

Anyone see a problem with having multiple working copies in the www testing directory.  One working copy for each individual (2 of us ATM)?

 

Nothing wrong with that as long as other people don't mess you working copy. It's up to you if you can afford this risk (or overcome it using access privileges).

[dardub]

Thanks for the advice. I don't think we'll have a problem with accidentally working on the wrong copy. However, if more people start working on it, then yes it could get messy.

[Mchl]

There's also a possiblilty for a single developer to have multiple working cpies checked out. Nothing wrong with that, although it might indicate the need to create one or more branches.

[roopurt18]

The other local server we have is a windows domain server with a local and offsite backup.  Would you recommend using the windows server with virtual svn server?

I don't know what you mean by "using windows server with virtual svn server."  Just install SVN on the windows machine.

 

Or putting svn server on our staging server?

If you're using staging as synonymous with production, then no do not put SVN on the production server.

 

In your case I'd use the Windows Domain Server as the SVN server as well.

[ignace]

development server: php, apache, mysql, phpunit, svn, ..

testing server: phpunit, phpundercontrol, ..

production server: php, apache, mysql

staging server: exact copy of production server

[roopurt18]

I'll say it again: SVN doesn't belong on the development server.

[ignace]

I'll say it again: SVN doesn't belong on the development server.

 

There are different setups, currently my desktop acts as the development server. If I were to work with multiple developers then the development server would move to a centralized server and SVN would not be present as it wouldn't be used.

[roopurt18]

I don't see why you wouldn't use SVN with multiple developers.  Regardless of the number of developers, some form of version control should always be used.

 

What I'm saying is SVN should not be hosted on a development server.  A development server is the machine a developer uses to write untested, new code or to modify existing code (again untested).  Storing your SVN repos on a development server is a huge risk in that a developer might accidentally hose the server and all of the company's source code could be lost.  That would be nothing short of disastrous.

 

Developers develop on their own machines or on development servers.

Code is checked into the repository which resides on another production server within the company that is closed to the public.

Code is tested on some machine, which could be a development server or a staging server.

Code that passes testing is checked into version control as a release.

The release is exported to the production server which is providing via hosting account or a DMZ within the company (or via some other mechanism).

 

It is foolish to develop code on a machine which holds the company's bread and butter.  It's also foolish to store the company's bread and butter on a server that is open to the public (i.e. on a web server).  Do so at your own peril. 

[Mchl]

And it's also foolish to keep your bread and butter on a machine you take out with you anywhere you go... I'm foolish

[dardub]

 

@Roopert

I don't know what you mean by "using windows server with virtual svn server."  Just install SVN on the windows machine.

 

Sorry, I meant Visual SVN not Virtual SVN.  That's definitely a confusing statement. Visual SVN is an svn server for windows if you're not aware.

 

Yes, I don't see the point in stopping version control because you are working with other developers.  That's one of the reasons I want to use svn, is to work with other developers.

 

I initially wanted to put the repo on the testing server because it's local and behind a firewall, but I understand how putting something that valuable on a potentially unstable server can be hazardous.

[dardub]

I guess I a reason I was hesitant to use the windows server for svn is because I didn't set up either of the local servers.  For one I am afraid of screwing something up on our domain server, and also I don't know how easily I'll be able to connect to the repo on the windows server from the linux testing server.

[andrewgauger]

I'd suggest a NAS (RAID 1 minimum) as your SVN if you put it on a box that has other roles.  This way, you can access the files if the server goes down.  (this is a design consideration not coming from someone who has a svn, so take it with the appropriate amount of salt).

[Mchl]

I don't know how easily I'll be able to connect to the repo on the windows server from the linux testing server.

 

I don't know Visual SVN. I use the one from CollabNet. There are no problems in connecting to it from outside

[ignace]

Storing your SVN repos on a development server is a huge risk in that a developer might accidentally hose the server and all of the company's source code could be lost.

 

I meant the SVN client not the server version. I forgot to mention that SVN (server binaries) is hosted on a separate server.