Apache slowly consumes all system memory while running streaming CGI

gattusunil · September 23, 2008

Hi All,

I have CGI where on request it streams all the files one by one to the client. The total streamed file sizes may vary from 30MB to 900MB. Apache consumes lot of memory while streaming files in the order clinet requested (esp., if the over all stream size is > 400MB) and finally i get OOM killing httpd and some times my system crashes.

Any help is appreciated very much.. As i am not able to find any solution for this.. and i am thinking of alternate server like GoAhead for Embedded Domain.

I am using the latest APACHE Version-2.2.9 with the following http.conf

##

## httpd.conf -- Apache HTTP server configuration file

##

#

# Based upon the NCSA server configuration files originally by Rob McCool.

#

# This is the main Apache server configuration file. It contains the

# configuration directives that give the server its instructions.

# See <URL:http://www.apache.org/docs/> for detailed information about

# the directives.

#

# Do NOT simply read the instructions in here without understanding

# what they do. They're here only as hints or reminders. If you are unsure

# consult the online docs. You have been warned.

#

# After this file is processed, the server will look for and process

# /usr/conf/srm.conf and then /usr/conf/access.conf

# unless you have overridden these with ResourceConfig and/or

# AccessConfig directives here.

#

# The configuration directives are grouped into three basic sections:

# 1. Directives that control the operation of the Apache server process as a

# whole (the 'global environment').

# 2. Directives that define the parameters of the 'main' or 'default' server,

# which responds to requests that aren't handled by a virtual host.

# These directives also provide default values for the settings

# of all virtual hosts.

# 3. Settings for virtual hosts, which allow Web requests to be sent to

# different IP addresses or hostnames and have them handled by the

# same Apache server process.

#

# Configuration and logfile names: If the filenames you specify for many

# of the server's control files begin with "/" (or "drive:/" for Win32), the

# server will use that explicit path. If the filenames do *not* begin

# with "/", the value of ServerRoot is prepended -- so "logs/foo.log"

# with ServerRoot set to "/usr/local/apache" will be interpreted by the

# server as "/usr/local/apache/logs/foo.log".

#

### Section 1: Global Environment

#

# The directives in this section affect the overall operation of Apache,

# such as the number of concurrent requests it can handle or where it

# can find its configuration files.

#

# ServerRoot: The top of the directory tree under which the server's

# configuration, error, and log files are kept.

#

# NOTE! If you intend to place this on an NFS (or otherwise network)

# mounted filesystem then please read the LockFile documentation

# (available at <URL:http://www.apache.org/docs/mod/core.html#lockfile>);

# you will save yourself a lot of trouble.

#

#ServerRoot "/etc/apache"

#

# The LockFile directive sets the path to the lockfile used when Apache

# is compiled with either USE_FCNTL_SERIALIZED_ACCEPT or

# USE_FLOCK_SERIALIZED_ACCEPT. This directive should normally be left at

# its default value. The main reason for changing it is if the logs

# directory is NFS mounted, since the lockfile MUST BE STORED ON A LOCAL

# DISK. The PID of the main server process is automatically appended to

# the filename.

#

#LockFile /var/run/httpd.lock

#

# PidFile: The file in which the server should record its process

# identification number when it starts.

#

PidFile /var/run/httpd.pid

#

# ScoreBoardFile: File used to store internal server process information.

# Not all architectures require this. But if yours does (you'll know because

# this file will be created when you run Apache) then you *must* ensure that

# no two invocations of Apache share the same scoreboard file.

#

ScoreBoardFile /var/run/httpd.scoreboard

#

# In the standard configuration, the server will process httpd.conf (this

# file, specified by the -f command line option), srm.conf, and access.conf

# in that order. The latter two files are now distributed empty, as it is

# recommended that all directives be kept in a single file for simplicity.

# The commented-out values below are the built-in defaults. You can have the

# server ignore these files altogether by using "/dev/null" (for Unix) or

# "nul" (for Win32) for the arguments to the directives.

#

#ResourceConfig conf/srm.conf

#AccessConfig conf/access.conf

#

# Timeout: The number of seconds before receives and sends time out.

#

Timeout 300

#

# KeepAlive: Whether or not to allow persistent connections (more than

# one request per connection). Set to "Off" to deactivate.

#

KeepAlive On

#

# MaxKeepAliveRequests: The maximum number of requests to allow

# during a persistent connection. Set to 0 to allow an unlimited amount.

# We recommend you leave this number high, for maximum performance.

#

MaxKeepAliveRequests 100

#

# KeepAliveTimeout: Number of seconds to wait for the next request from the

# same client on the same connection.

#

KeepAliveTimeout 15

#

# Server-pool size regulation. Rather than making you guess how many

# server processes you need, Apache dynamically adapts to the load it

# sees --- that is, it tries to maintain enough server processes to

# handle the current load, plus a few spare servers to handle transient

# load spikes (e.g., multiple simultaneous requests from a single

# Netscape browser).

#

# It does this by periodically checking how many servers are waiting

# for a request. If there are fewer than MinSpareServers, it creates

# a new spare. If there are more than MaxSpareServers, some of the

# spares die off. The default values are probably OK for most sites.

#

MinSpareServers 1

MaxSpareServers 2

#

# Number of servers to start initially --- should be a reasonable ballpark

# figure.

#

StartServers 2

#

# Limit on total number of servers running, i.e., limit on the number

# of clients who can simultaneously connect --- if this limit is ever

# reached, clients will be LOCKED OUT, so it should NOT BE SET TOO LOW.

# It is intended mainly as a brake to keep a runaway server from taking

# the system with it as it spirals down...

#

MaxClients 150

#

# MaxRequestsPerChild: the number of requests each child process is

# allowed to process before the child dies. The child will exit so

# as to avoid problems after prolonged use when Apache (and maybe the

# libraries it uses) leak memory or other resources. On most systems, this

# isn't really needed, but a few (such as Solaris) do have notable leaks

# in the libraries. For these platforms, set to something like 10000

# or so; a setting of 0 means unlimited.

#

# NOTE: This value does not include keepalive requests after the initial

# request per connection. For example, if a child process handles

# an initial request and 10 subsequent "keptalive" requests, it

# would only count as 1 request towards this limit.

#

MaxRequestsPerChild 1

#

# Listen: Allows you to bind Apache to specific IP addresses and/or

# ports, in addition to the default. See also the <VirtualHost>

# directive.

#

#Listen 3000

#Listen 12.34.56.78:80

#

# BindAddress: You can support virtual hosts with this option. This directive

# is used to tell the server which IP address to listen to. It can either

# contain "*", an IP address, or a fully qualified Internet domain name.

# See also the <VirtualHost> and Listen directives.

#

#BindAddress *

#

# Dynamic Shared Object (DSO) Support

#

# To be able to use the functionality of a module which was built as a DSO you

# have to place corresponding `LoadModule' lines at this location so the

# directives contained in it are actually available _before_ they are used.

# Please read the file README.DSO in the Apache 1.3 distribution for more

# details about the DSO mechanism and run `httpd -l' for the list of already

# built-in (statically linked and thus always available) modules in your httpd

# binary.

#

# Note: The order in which modules are loaded is important. Don't change

# the order below without expert advice.

#

# Example:

# LoadModule foo_module libexec/mod_foo.so

# Reconstruction of the complete module list from all available modules

# (static and shared ones) to achieve correct module execution order.

# [WHENEVER YOU CHANGE THE LOADMODULE SECTION ABOVE UPDATE THIS, TOO]

#

# ExtendedStatus controls whether Apache will generate "full" status

# information (ExtendedStatus On) or just basic information (ExtendedStatus

# Off) when the "server-status" handler is called. The default is Off.

#

#ExtendedStatus On

### Section 2: 'Main' server configuration

#

# The directives in this section set up the values used by the 'main'

# server, which responds to any requests that aren't handled by a

# <VirtualHost> definition. These values also provide defaults for

# any <VirtualHost> containers you may define later in the file.

#

# All of these directives may appear inside <VirtualHost> containers,

# in which case these default settings will be overridden for the

# virtual host being defined.

#

# If your ServerType directive (set earlier in the 'Global Environment'

# section) is set to "inetd", the next few directives don't have any

# effect since their settings are defined by the inetd configuration.

# Skip ahead to the ServerAdmin directive.

#

# Port: The port to which the standalone server listens. For

# ports < 1023, you will need httpd to be run as root initially.

#

Listen 80

#

# If you wish httpd to run as a different user or group, you must run

# httpd as root initially and it will switch.

#

# User/Group: The name (or #number) of the user/group to run httpd as.

# . On SCO (ODT 3) use "User nouser" and "Group nogroup".

# . On HPUX you may not be able to use shared memory as nobody, and the

# suggested workaround is to create a user www and use that user.

# NOTE that some kernels refuse to setgid(Group) or semctl(IPC_SET)

# when the value of (unsigned)Group is above 60000;

# don't use Group nobody on these systems!

#

User www

Group www

#

# ServerAdmin: Your address, where problems with the server should be

# e-mailed. This address appears on some server-generated pages, such

# as error documents.

#

ServerAdmin root@localhost

#

# ServerName allows you to set a host name which is sent back to clients for

# your server if it's different than the one the program would get (i.e., use

# "www" instead of the host's real name).

#

# Note: You cannot just invent host names and hope they work. The name you

# define here must be a valid DNS name for your host. If you don't understand

# this, ask your network administrator.

# If your host doesn't have a registered DNS name, enter its IP address here.

# You will have to access it by its address (e.g., http://123.45.67.89/)

# anyway, and this will make redirections work in a sensible way.

#

# 127.0.0.1 is the TCP/IP local loop-back address, often named localhost. Your

# machine always knows itself by this address. If you use Apache strictly for

# local testing and development, you may use 127.0.0.1 as the server name.

#

ServerName localhost

#

# DocumentRoot: The directory out of which you will serve your

# documents. By default, all requests are taken from this directory, but

# symbolic links and aliases may be used to point to other locations.

#

DocumentRoot "/var/www/html/"

#

# Each directory to which Apache has access, can be configured with respect

# to which services and features are allowed and/or disabled in that

# directory (and its subdirectories).

#

# First, we configure the "default" to be a very restrictive set of

# permissions.

#

Options FollowSymLinks

AllowOverride None

</Directory>

#

# Note that from this point forward you must specifically allow

# particular features to be enabled - so if something's not working as

# you might expect, make sure that you have specifically enabled it

# below.

#

# This should be changed to whatever you set DocumentRoot to.

#

#

# This may also be "None", "All", or any combination of "Indexes",

# "Includes", "FollowSymLinks", "ExecCGI", or "MultiViews".

#

# Note that "MultiViews" must be named *explicitly* --- "Options All"

# doesn't give it to you.

#

Options Indexes FollowSymLinks MultiViews

#

# This controls which options the .htaccess files in directories can

# override. Can also be "All", or any combination of "Options", "FileInfo",

# "AuthConfig", and "Limit"

#

AllowOverride None

#

# Controls who can get stuff from this server.

#

Order allow,deny

Allow from all

</Directory>

#

# UserDir: The name of the directory which is appended onto a user's home

# directory if a ~user request is received.

#

UserDir public_html

</IfModule>

#

# Control access to UserDir directories. The following is an example

# for a site where these directories are restricted to read-only.

#

#<Directory /home/*/public_html>

# AllowOverride FileInfo AuthConfig Limit

# Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec

# <Limit GET POST OPTIONS PROPFIND>

# Order allow,deny

# Allow from all

# </Limit>

# <LimitExcept GET POST OPTIONS PROPFIND>

# Order deny,allow

# Deny from all

# </LimitExcept>

#</Directory>

#

# DirectoryIndex: Name of the file or files to use as a pre-written HTML

# directory index. Separate multiple entries with spaces.

#

DirectoryIndex index.html

</IfModule>

#

# AccessFileName: The name of the file to look for in each directory

# for access control information.

#

AccessFileName .htaccess

#

# The following lines prevent .htaccess files from being viewed by

# Web clients. Since .htaccess files often contain authorization

# information, access is disallowed for security reasons. Comment

# these lines out if you want Web visitors to see the contents of

# .htaccess files. If you change the AccessFileName directive above,

# be sure to make the corresponding changes here.

#

# Also, folks tend to use names such as .htpasswd for password

# files, so this will protect those as well.

#

Order allow,deny

Deny from all

</Files>

#

# CacheNegotiatedDocs: By default, Apache sends "Pragma: no-cache" with each

# document that was negotiated on the basis of content. This asks proxy

# servers not to cache the document. Uncommenting the following line disables

# this behavior, and proxies will be allowed to cache the documents.

#

#CacheNegotiatedDocs

#

# UseCanonicalName: (new for 1.3) With this setting turned on, whenever

# Apache needs to construct a self-referencing URL (a URL that refers back

# to the server the response is coming from) it will use ServerName and

# Port to form a "canonical" name. With this setting off, Apache will

# use the hostname:port that the client supplied, when possible. This

# also affects SERVER_NAME and SERVER_PORT in CGI scripts.

#

UseCanonicalName On

#

# TypesConfig describes where the mime.types file (or equivalent) is

# to be found.

#

TypesConfig /etc/apache/mime.types

</IfModule>

#

# DefaultType is the default MIME type the server will use for a document

# if it cannot otherwise determine one, such as from filename extensions.

# If your server contains mostly text or HTML documents, "text/plain" is

# a good value. If most of your content is binary, such as applications

# or images, you may want to use "application/octet-stream" instead to

# keep browsers from trying to display binary files as though they are

# text.

#

DefaultType text/plain

#

# The mod_mime_magic module allows the server to use various hints from the

# contents of the file itself to determine its type. The MIMEMagicFile

# directive tells the module where the hint definitions are located.

# mod_mime_magic is not part of the default server (you have to add

# it yourself with a LoadModule [see the DSO paragraph in the 'Global

# Environment' section], or recompile the server and include mod_mime_magic

# as part of the configuration), so it's enclosed in an <IfModule> container.

# This means that the MIMEMagicFile directive will only be processed if the

# module is part of the server.

#

MIMEMagicFile /etc/apache/magic

</IfModule>

#

# HostnameLookups: Log the names of clients or just their IP addresses

# e.g., www.apache.org (on) or 204.62.129.132 (off).

# The default is off because it'd be overall better for the net if people

# had to knowingly turn this feature on, since enabling it means that

# each client request will result in AT LEAST one lookup request to the

# nameserver.

#

HostnameLookups Off

#

# ErrorLog: The location of the error log file.

# If you do not specify an ErrorLog directive within a <VirtualHost>

# container, error messages relating to that virtual host will be

# logged here. If you *do* define an error logfile for a <VirtualHost>

# container, that host's errors will be logged there and not here.

#

ErrorLog /tmp/error_log

#

# LogLevel: Control the number of messages logged to the error_log.

# Possible values include: debug, info, notice, warn, error, crit,

# alert, emerg.

#

LogLevel warn

#

# The following directives define some format nicknames for use with

# a CustomLog directive (see below).

#

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

LogFormat "%h %l %u %t \"%r\" %>s %b" common

LogFormat "%{Referer}i -> %U" referer

LogFormat "%{User-agent}i" agent

#

# The location and format of the access logfile (Common Logfile Format).

# If you do not define any access logfiles within a <VirtualHost>

# container, they will be logged here. Contrariwise, if you *do*

# define per-<VirtualHost> access logfiles, transactions will be

# logged therein and *not* in this file.

#

CustomLog /tmp/access_log common

#

# If you would like to have agent and referer logfiles, uncomment the

# following directives.

#

#CustomLog /var/log/apache/referer_log referer

#CustomLog /var/log/apache/agent_log agent

#

# If you prefer a single logfile with access, agent, and referer information

# (Combined Logfile Format) you can use the following directive.

#

#CustomLog /var/log/apache/access_log combined

#

# Optionally add a line containing the server version and virtual host

# name to server-generated pages (error documents, FTP directory listings,

# mod_status and mod_info output etc., but not CGI generated documents).

# Set to "EMail" to also include a mailto: link to the ServerAdmin.

# Set to one of: On | Off | EMail

#

ServerSignature On

#

# Aliases: Add here as many aliases as you need (with no limit). The format is

# Alias fakename realname

#

#

# Note that if you include a trailing / on fakename then the server will

# require it to be present in the URL. So "/icons" isn't aliased in this

# example, only "/icons/"..

#

Alias /icons/ "/var/www/html/icons/"

Options Indexes MultiViews

AllowOverride None

Order allow,deny

Allow from all

</Directory>

#

# ScriptAlias: This controls which directories contain server scripts.

# ScriptAliases are essentially the same as Aliases, except that

# documents in the realname directory are treated as applications and

# run by the server when requested rather than as documents sent to the client.

# The same rules about trailing "/" apply to ScriptAlias directives as to

# Alias.

#

ScriptAlias /cgi-bin/ "/var/www/html/cgi-bin/"

#

# "/home/httpd/html/cgi-bin" should be changed to whatever your ScriptAliased

# CGI directory exists, if you have that configured.

#

AllowOverride None

Options None

Order allow,deny

Allow from all

</Directory>

</IfModule>

# End of aliases.

#

# Redirect allows you to tell clients about documents which used to exist in

# your server's namespace, but do not anymore. This allows you to tell the

# clients where to look for the relocated document.

# Format: Redirect old-URI new-URL

#

# Directives controlling the display of server-generated directory listings.

#

#

# FancyIndexing is whether you want fancy directory indexing or standard

#

IndexOptions FancyIndexing

#

# AddIcon* directives tell the server which icon to show for different

# files or filename extensions. These are only displayed for

# FancyIndexed directories.

#

AddIconByEncoding (CMP,/icons/compressed.gif) x-compress x-gzip

AddIconByType (TXT,/icons/text.gif) text/*

AddIconByType (IMG,/icons/image2.gif) image/*

AddIconByType (SND,/icons/sound2.gif) audio/*

AddIconByType (VID,/icons/movie.gif) video/*

AddIcon /icons/binary.gif .bin .exe

AddIcon /icons/binhex.gif .hqx

AddIcon /icons/tar.gif .tar

AddIcon /icons/world2.gif .wrl .wrl.gz .vrml .vrm .iv

AddIcon /icons/compressed.gif .Z .z .tgz .gz .zip

AddIcon /icons/a.gif .ps .ai .eps

AddIcon /icons/layout.gif .html .shtml .htm .pdf

AddIcon /icons/text.gif .txt

AddIcon /icons/c.gif .c

AddIcon /icons/p.gif .pl .py

AddIcon /icons/f.gif .for

AddIcon /icons/dvi.gif .dvi

AddIcon /icons/uuencoded.gif .uu

AddIcon /icons/script.gif .conf .sh .shar .csh .ksh .tcl

AddIcon /icons/tex.gif .tex

AddIcon /icons/bomb.gif core

AddIcon /icons/back.gif ..

AddIcon /icons/hand.right.gif README

AddIcon /icons/folder.gif ^^DIRECTORY^^

AddIcon /icons/blank.gif ^^BLANKICON^^

#

# DefaultIcon is which icon to show for files which do not have an icon

# explicitly set.

#

DefaultIcon /icons/unknown.gif

#

# AddDescription allows you to place a short description after a file in

# server-generated indexes. These are only displayed for FancyIndexed

# directories.

# Format: AddDescription "description" filename

#

#AddDescription "GZIP compressed document" .gz

#AddDescription "tar archive" .tar

#AddDescription "GZIP compressed tar archive" .tgz

#

# ReadmeName is the name of the README file the server will look for by

# default, and append to directory listings.

#

# HeaderName is the name of a file which should be prepended to

# directory indexes.

#

# If MultiViews are amongst the Options in effect, the server will

# first look for name.html and include it if found. If name.html

# doesn't exist, the server will then look for name.txt and include

# it as plaintext if found.

#

ReadmeName README

HeaderName HEADER

#

# IndexIgnore is a set of filenames which directory indexing should ignore

# and not include in the listing. Shell-style wildcarding is permitted.

#

IndexIgnore .??* *~ *# HEADER* README* RCS CVS *,v *,t

</IfModule>

# End of indexing directives.

#

# Document types.

#

#

# AddEncoding allows you to have certain browsers (Mosaic/X 2.1+) uncompress

# information on the fly. Note: Not all browsers support this.

# Despite the name similarity, the following Add* directives have nothing

# to do with the FancyIndexing customization directives above.

#

AddEncoding x-compress Z

AddEncoding x-gzip gz tgz

#

# AddLanguage allows you to specify the language of a document. You can

# then use content negotiation to give a browser a file in a language

# it can understand.

#

# Note 1: The suffix does not have to be the same as the language

# keyword --- those with documents in Polish (whose net-standard

# language code is pl) may wish to use "AddLanguage pl .po" to

# avoid the ambiguity with the common suffix for perl scripts.

#

# Note 2: The example entries below illustrate that in quite

# some cases the two character 'Language' abbreviation is not

# identical to the two character 'Country' code for its country,

# E.g. 'Danmark/dk' versus 'Danish/da'.

#

# Note 3: In the case of 'ltz' we violate the RFC by using a three char

# specifier. But there is 'work in progress' to fix this and get

# the reference data for rfc1766 cleaned up.

#

# Danish (da) - Dutch (nl) - English (en) - Estonian (ee)

# French (fr) - German (de) - Greek-Modern (el)

# Italian (it) - Korean (kr) - Norwegian (no)

# Portugese (pt) - Luxembourgeois* (ltz)

# Spanish (es) - Swedish (sv) - Catalan (ca) - Czech(cz)

# Polish (pl) - Brazilian Portuguese (pt-br) - Japanese (ja)

# Russian (ru)

#

AddLanguage da .dk

AddLanguage nl .nl

AddLanguage en .en

AddLanguage et .ee

AddLanguage fr .fr

AddLanguage de .de

AddLanguage el .el

AddLanguage he .he

AddCharset ISO-8859-8 .iso8859-8

AddLanguage it .it

AddLanguage ja .ja

AddCharset ISO-2022-JP .jis

AddLanguage kr .kr

AddCharset ISO-2022-KR .iso-kr

AddLanguage no .no

AddLanguage pl .po

AddCharset ISO-8859-2 .iso-pl

AddLanguage pt .pt

AddLanguage pt-br .pt-br

AddLanguage ltz .lu

AddLanguage ca .ca

AddLanguage es .es

AddLanguage sv .se

AddLanguage cz .cz

AddLanguage ru .ru

AddLanguage zh-tw .tw

AddLanguage tw .tw

AddCharset Big5 .Big5 .big5

AddCharset WINDOWS-1251 .cp-1251

AddCharset CP866 .cp866

AddCharset ISO-8859-5 .iso-ru

AddCharset KOI8-R .koi8-r

AddCharset UCS-2 .ucs2

AddCharset UCS-4 .ucs4

AddCharset UTF-8 .utf8

# LanguagePriority allows you to give precedence to some languages

# in case of a tie during content negotiation.

#

# Just list the languages in decreasing order of preference. We have

# more or less alphabetized them here. You probably want to change this.

#

LanguagePriority en da nl et fr de el it ja kr no pl pt pt-br ru ltz ca es sv tw

</IfModule>

#

# AddType allows you to tweak mime.types without actually editing it, or to

# make certain files to be certain types.

#

# For example, the PHP 3.x module (not part of the Apache distribution - see

# http://www.php.net) will typically use:

#

#AddType application/x-httpd-php3 .php3

#AddType application/x-httpd-php3-source .phps

#

# And for PHP 4.x, use:

#

#AddType application/x-httpd-php .php

#AddType application/x-httpd-php-source .phps

AddType application/x-tar .tgz

#

# AddHandler allows you to map certain file extensions to "handlers",

# actions unrelated to filetype. These can be either built into the server

# or added with the Action command (see below)

#

# If you want to use server side includes, or CGI outside

# ScriptAliased directories, uncomment the following lines.

#

# To use CGI scripts:

#

#AddHandler cgi-script .cgi

#

# To use server-parsed HTML files

#

#AddType text/html .shtml

#AddHandler server-parsed .shtml

#

# Uncomment the following line to enable Apache's send-asis HTTP file

# feature

#

#AddHandler send-as-is asis

#

# If you wish to use server-parsed imagemap files, use

#

#AddHandler imap-file map

#

# To enable type maps, you might want to use

#

#AddHandler type-map var

</IfModule>

# End of document types.

#

# Action lets you define media types that will execute a script whenever

# a matching file is called. This eliminates the need for repeated URL

# pathnames for oft-used CGI file processors.

# Format: Action media/type /cgi-script/location

# Format: Action handler-name /cgi-script/location

#

# MetaDir: specifies the name of the directory in which Apache can find

# meta information files. These files contain additional HTTP headers

# to include when sending the document

#

#MetaDir .web

#

# MetaSuffix: specifies the file name suffix for the file containing the

# meta information.

#

#MetaSuffix .meta

#

# Customizable error response (Apache style)

# these come in three flavors

#

# 1) plain text

#ErrorDocument 500 "The server made a boo boo.

# n.b. the single leading (") marks it as text, it does not get output

#

# 2) local redirects

#ErrorDocument 404 /missing.html

# to redirect to local URL /missing.html

#ErrorDocument 404 /cgi-bin/missing_handler.pl

# N.B.: You can redirect to a script or a document using server-side-includes.

#

# 3) external redirects

#ErrorDocument 402 http://some.other_server.com/subscription_info.html

# N.B.: Many of the environment variables associated with the original

# request will *not* be available to such a script.

#

# Customize behaviour based on the browser

#

#

# The following directives modify normal HTTP response behavior.

# The first directive disables keepalive for Netscape 2.x and browsers that

# spoof it. There are known problems with these browser implementations.

# The second directive is for Microsoft Internet Explorer 4.0b2

# which has a broken HTTP/1.1 implementation and does not properly

# support keepalive when it is used on 301 or 302 (redirect) responses.

#

BrowserMatch "Mozilla/2" nokeepalive

BrowserMatch "MSIE 4\.0b2;" nokeepalive downgrade-1.0 force-response-1.0

#

# The following directive disables HTTP/1.1 responses to browsers which

# are in violation of the HTTP/1.0 spec by not being able to grok a

# basic 1.1 response.

#

BrowserMatch "RealPlayer 4\.0" force-response-1.0

BrowserMatch "Java/1\.0" force-response-1.0

BrowserMatch "JDK/1\.0" force-response-1.0

</IfModule>

# End of browser customization directives

# Allow deflate operation for html and xml files

# added by Grant Irani

AddOutputFilterByType DEFLATE text/html text/plain text/xml text/javascript

AddOutputFilterByType DEFLATE application/x-unknown application/xml

AddOutputFilterByType DEFLATE application/octet_stream

</Location>

#

# Allow server status reports, with the URL of http://servername/server-status

# Change the ".your_domain.com" to match your domain to enable.

#

#<Location /server-status>

# SetHandler server-status

# Order deny,allow

# Deny from all

# Allow from .your_domain.com

#</Location>

#

# Allow remote server configuration reports, with the URL of

# http://servername/server-info (requires that mod_info.c be loaded).

# Change the ".your_domain.com" to match your domain to enable.

#

#<Location /server-info>

# SetHandler server-info

# Order deny,allow

# Deny from all

# Allow from .your_domain.com

#</Location>

#

# There have been reports of people trying to abuse an old bug from pre-1.1

# days. This bug involved a CGI script distributed as a part of Apache.

# By uncommenting these lines you can redirect these attacks to a logging

# script on phf.apache.org. Or, you can record them yourself, using the script

# support/phf_abuse_log.cgi.

#

#<Location /cgi-bin/phf*>

# Deny from all

# ErrorDocument 403 http://phf.apache.org/phf_abuse_log.cgi

#</Location>

#

# Proxy Server directives. Uncomment the following lines to

# enable the proxy server:

#

#<IfModule mod_proxy.c>

# ProxyRequests On

# <Directory proxy:*>

# Order deny,allow

# Deny from all

# Allow from .your_domain.com

# </Directory>

#

# Enable/disable the handling of HTTP/1.1 "Via:" headers.

# ("Full" adds the server version; "Block" removes all outgoing Via: headers)

# Set to one of: Off | On | Full | Block

#

# ProxyVia On

#

# To enable the cache as well, edit and uncomment the following lines:

# (no cacheing without CacheRoot)

#

# CacheRoot "/var/proxy"

# CacheSize 5

# CacheGcInterval 4

# CacheMaxExpire 24

# CacheLastModifiedFactor 0.1

# CacheDefaultExpire 1

# NoCache a_domain.com another_domain.edu joes.garage_sale.com

#</IfModule>

# End of proxy directives.

### Section 3: Virtual Hosts

#

# VirtualHost: If you want to maintain multiple domains/hostnames on your

# machine you can setup VirtualHost containers for them. Most configurations

# use only name-based virtual hosts so the server doesn't need to worry about

# IP addresses. This is indicated by the asterisks in the directives below.

#

# Please see the documentation at <URL:http://www.apache.org/docs/vhosts/>

# for further details before you try to setup virtual hosts.

#

# You may use the command line option '-S' to verify your virtual host

# configuration.

#

# Use name-based virtual hosting.

#

#NameVirtualHost *

#

# VirtualHost example:

# Almost any Apache directive may go into a VirtualHost container.

# The first VirtualHost section is used for requests without a known

# server name.

#

#<VirtualHost *>

# ServerAdmin webmaster@dummy-host.example.com

# DocumentRoot /www/docs/dummy-host.example.com

# ServerName dummy-host.example.com

# ErrorLog logs/dummy-host.example.com-error_log

# CustomLog logs/dummy-host.example.com-access_log common

#</VirtualHost>

steviewdr · September 23, 2008

Could it be the cgi script at fault?

Enabling server-status and server-info might shed some light on whats exactly happening.

-steve

gattusunil · September 23, 2008

No-Steve.. Actually if i give the same request from CURL with compressed format, apache consumes only 3% of memory while downloading 600MB data.

But if i give the same without compressed, it sucks my memory for 600MB data.. i can assure def.. its not a CGI fault..

Can u guide me how to enable server info to proceed further as per ur suggestion..

steviewdr · September 23, 2008

See the following http.conf config:

http://www.uow.edu.au/~nabg/WebServer/Chapter3/WindowsConf.html

Do a search for server-info and server-status.

You will also need to uncoment the mod_status module as in:

#LoadModule status_module modules/ApacheModuleStatus.dll

Can you outline what you are doing with curl?

Test 1: mod_gzip on apache enabled?

Test 2: no mod_gzip, and it uses up memory?

Also if you had the script, I could test it on a typical server.

-steve

gattusunil · September 24, 2008

Ok.. Steve Let me explain the Full Story !!

1. My Environment is Power PC with Montavista Linux.

2. I have cross compiled the latest version of Apache with the follownig options

./configure --prefix=/etc/apache --host=ppc ac_cv_func_setpgrp_void=yes ac_cv_file__dev_zero=no ac_cv_define_PTHREAD_PROCESS_SHARED=no apr_cv_mutex_robust_shared=no ac_cv_tcp_nodelay_inherited=no apr_cv_tcp_nodelay_with_cork=no ac_cv_file_dbd_apr_dbd_mysql_c=no ap_void_ptr_lt_long=no ac_cv_exeext=no --enable-static-support --enable-ssl --enable-deflate --with-krb5=no

Please let me know if i did any mistake while configuring the apache

3. As you can see i have compiled the apache with mod deflate option.

I request the apache from client with compression with the following option

1. mod_gzip on apache enabled with this command line option--> curl --compressed "http://192.158.145.122/cgi-bin/record.cgi?cmd=GET_RECORD&record_type=r1&start_time=1221304140&stop_time=1221446760&format=binary" >/dev/null

I request the apache from client without compression with the following option

2. no mod_gzip on apache enabled with this command line option--> curl -k -v "http://192.158.145.122/cgi-bin/record.cgi?cmd=GET_RECORD&record_type=r1&start_time=1221304140&stop_time=1221446760&format=binary" >/dev/null

Now My CGI invokes one more libray API which does opening a file, wrting to STDOUT, and closing that file.

The no. of files to be streamed will be decided based upon the start_time & stop_time received from the Key Value pairs in the request. Normally my application will form files for every minute with 600KB data. If I request for 10 Hrs data (10*60*600KB=120000 KB) 360 MB data will be streamed by opening, writing to STDOUT & closing the files for 600 times (100*60).

My CGI script may not work straight away for you as it has got lot of dependencies which are for the board specific. So general way of simulation would be CGI doing this Job (o/w/c for 600 files .. can be the same file with 600KB data). Hope i am clear what i am doing Steve.

Please let me know if you need any clarifications further

gattusunil · September 24, 2008

Steve -- Can i get ur personal email-id as this is very urgent for me..or else u can reach me at gattusunil@yahoo.co.in

Sign In

Apache slowly consumes all system memory while running streaming CGI

Recommended Posts

gattusunil

Link to comment

Share on other sites

steviewdr

Link to comment

Share on other sites

gattusunil

Link to comment

Share on other sites

steviewdr

Link to comment

Share on other sites

gattusunil

Link to comment

Share on other sites

gattusunil

Link to comment

Share on other sites

Join the conversation

Browse

Activity

Important Information