Finding Apache Average Memory Usage on Debian Wheezy

I wanted to find the average process memory usage on Debian wheezy to calculate MaxClients.

I used this script to do it:

sudo ps -ef | grep apache2 | grep -v ^$USER | awk '{ print $2 '} | xargs sudo pmap -d | grep ^mapped: | awk '{ print $4 }' | cut -dK -f1 | awk '{SUM += $1} END { print SUM/NR }'

Quick explanation, starts out with a process list, filters for apache2, removes the grep command from the list, prints the second column, uses pmap to display memory usage, then grep to filter out the mapped memory. Again printing the fourth column, then using cut to remove the K, and awk to sum up the records. NR in awk is the number of input records seen so far.

To get MaxClients divide your total memory (minus some margin for other system processes) by the average process memory usage to get the number for MaxClients.

Example, 50K process size, memory of 1.8G, 1.8G/50K = 36

Note you shouldn’t use the ps VSZ as it includes shared memory.

Install Postgresql 9.2 on Debian Squeeze

I was getting the following errors because the squeeze packages are too old for 9.2:

The following packages have unmet dependencies:
 postgresql-9.1 : Depends: libpq5 (>= 9.1~) but 8.4.13-0squeeze1 is to be installed
                  Depends: postgresql-client-9.1 but it is not going to be installed
                  Depends: postgresql-common (>= 115~) but 113+squeeze1 is to be installed

These are the steps I took to get postgresql 9.2 running on debian squeeze:

Add backports to /etc/apt/sources.list:

deb http://backports.debian.org/debian-backports squeeze-backports main
sudo apt-get update
sudo apt-get -t squeeze-backports install libpq5 postgresql-common

Add the postgresql repo to /etc/apt/sources.list:

deb http://pgapt.debian.net/ squeeze-pgdg main 9.0 9.2
sudo apt-get update
sudo apt-get install postgresql-9.2

Instructions have changed a bit, check out https://wiki.postgresql.org/wiki/Apt

Making Drupal 7 ViewsSlideshow Pager Clickable

While setting up a homepage slider, I used a modified version of the solutions found here. The solution provided worked great, except I needed a clickable version.

Create a new template called views-slideshow-slide-counter.tpl.php (notice that I changed it from above as I needed the ID to make the click work):

<?php $slide_count = count($view->result); for($i=1;$i<=$slide_count;++$i) { $n = $i - 1; $active = $i == 1 ? 'active' : ''; print '  '; } ?>

In application.js:

(function($) {
  $(document).ready(function() {

    Drupal.viewsSlideshowSlideCounter.transitionBegin = function (options) {
      $('#views_slideshow_slide_counter_' + options.slideshowID + ' span').removeClass('active');
      $('#views_slideshow_slide_counter_' + options.slideshowID + ' .num' + (options.slideNum + 1)).addClass('active');
    };

    $('.slide-block').click(function() {
      var id = $(this).attr('id').split('-').pop();
      var options = {'action': 'goToSlide', 'slideshowID': 'homepage_slider-block', 'slideNum': parseInt(id)};
      Drupal.viewsSlideshow.action(options);
    });

  });
})(jQuery);

Hope this helps someone trying to figure out how to make the pager clickable.

Superbowl 0-9 Pool Last Digit Frequency

If you’re in a 0-9 pool, where the total of the game scores last digit determines if you win, you may be interested in the history of your number. Here is a plot showing the frequency of the last digit from superbowls between 1968-2011.

Creating snapshots on AWS with ruby and awstools

Since the recent outage, I’ve been looking for simple scripts to take a snapshot of a volume and only keep a certain number of these snapshots around. I wanted something that I could run from cron in different intervals for different volumes. I found a few scripts lying around, none of which I really liked so I decided to put together one in ruby.

I created the awstools gem, submitted a patch to amazon-ec2 to allow filter requests on the snapshot api and now I have the tool I was looking for.

Check it out here and let me know what you think.

Until my patch is accepted you’ll have to use my fork of the amazon-ec2 gem. Which means you’ll have to clone it to your machine, build the gem with “gem build amazon-ec2.gemspec” then install that gem with “gem install amazon-ec2…”.

To install awstools

gem install awstools

Or to build it yourself, clone the repo to your machine, and then run:

cd awstools
bundle
gem build awstools.gemspec
gem install awstools-0.0.1.gem

Once installed you can run something like the following from cron:

0 0 * * * snapshots -v volume-id -d "Some Description" -k 2

That will take a snapshot once a day and only keep two snapshots on hand.

Note that there is an -s flag to simulate, so you can see what it will actually do without creating or deleting any snapshots.

If you see all the volumes at amazon fly by, its because you aren’t using my fork of amazon-ec2.

Using emacs to post to your blog

I’ve recently started playing with emacs (again). One feature that I just tried out is weblogger mode. It allows you to post entries to your blog directly from emacs using xmlrpc. Very simple setup. Since I’m using elpa, I can do a M-x package-list-packages, select weblogger to install and install it. Once installed you simply run M-x weblogger-setup-weblog, provide the credentials and are connect to your blog.

Check out the docs over at the Emacs Wiki

To be honest, the only reason I wrote the post was to test posting from emacs!

Autotest keeps running

I had an issue with autotest where the tests would just keep running, even when all the tests passed. After doing some searching I found the following code helpful from this post:

Put this in your ~/.autotest, leave out the growl line if you’re not using that.

require 'autotest/growl'

Autotest.add_hook :initialize do |at|
  %w{.git vendor rerun.txt}.each {|exception| at.add_exception(exception)}
end

When I initially tried this code it didn’t work. To figure out why autotest thought it had to keep running I ran the following:

find ./ -cmin -1 

This will show you any files that changed in the last minute. I had to add db, log, system and public to the list above. Depending on your configuration you might run into other files or folders that need to be excluded.

Issue using insert_after/before “ActionDispatch::Static”

We are currently building a new product on Rails 3 with a couple custom middleware rack apps. Everything is working great on development and we’re progressing nicely. Today I decided to start setting up the production environment and the following error popped up:

No such middleware to insert after: "ActionDispatch::Static"

I had this in my config/application.rb:

config.middleware.insert_after('ActionDispatch::Static', '::API::Throttle')

I received this error while trying to run rake or startup unicorn. After looking around trying to figure out what I was doing wrong I discovered this in the production config.

config.serve_static_assets = false

Essentially what this does is remove “ActionDispatch::Static” from the rails middleware stack. I had no idea that I could not rely on that being there.

Solution, use insert_before ‘Rack::Lock’ instead:

config.middleware.insert_before('Rack::Lock', '::API::Throttle')

Hope this saves someone some time!

Language Mismatch

One of the reasons that I believe communication between developers and non-developers is difficult is because developers must work in a world of specifics and details whereas a non-developer can casually neglect those details without even knowing they’ve been neglected.

For instance a non-developer says we need to allow users to log into this system. Easy enough, non-developer thinks his part is done. The developer looks at this requirement and thinks, is it a single users system? Will people register for this system? Is htaccess acceptable? Should it be a web based login? Etc. etc.

The above example is overly simplified and does not entirely show the mismatch. The only way to improve communications between the two groups is to ensure both groups understand how the other thinks. Without this understanding there will be tension that doesn’t necessarily need to exist.

The non-developer has most likely been educated in a less scientific and more abstract manner. This leads the non-developer to neglect details that are essential to any system designed to run on a computer that has to understand all paths through the system.

The developer on the other hand has to think about and understand all these paths to be able to build a system that fully functions. Always thinking about and contemplating whether or not the requirements are complete enough and whether they conflict with each other.

I believe the only way to resolve this mismatch is through educating both parties to be more understand of how the other must think. With this education maybe each party can move toward the others way of thinking enough to make the resulting system that much better.

Some VIM Commands

Every once in a while I try to review some vim information to get additional practice with the editor. Today I reviewed file buffers, visual mode and markers. All very useful in editing files with vim.

File buffers are great when editing more than file.
:buffers
:b1 :b2 :b3
:bn :bp
:sbn :sbp

Visual Mode
In normal mode use v, shift-v or ctl-v (block edit)
Use >, <, r, c, d, etc.

Markers
In normal mode use ma, mb, mc, etc
Then use ‘a, ‘b, ‘c etc.

Windows
:sp, :open, ctl-Wn, etc.
ctl-w, ctl-w to switch windows
Buffer commands above come in useful here

Completion, Omni-completion
ctl-p, ctl-n to complete with words from file
ctl-x, ctl-o to use syntax completion
Very useful things in a text editor!