Ruby to the Rescue!
Lately at my work we’ve been having some real problems with a clients servers, basically PHP has been seg faulting during memory allocation (for an as yet unknown reason) and apache processes keep getting stuck because of it.
So far the client is refusing to do anything to fix it at the moment (they dont want to screw with the servers this close to christmas), so that leaves me and a couple of the other guys having to go through all 9 servers every couple of hours to try and find which server has seg faulted, and then restart the stuck apache process, a little tedious i think you’ll agree.
So last night i decided to hack up a little ruby script to do the hunting work for us and i thought id share it with you guys 
require 'rubygems' require 'net/ssh' hosts = %w( web-server.test.com ) puts "--------------------------------" hosts.each do |host| Net::SSH.start(host,"********",:password => "********", :port => 22) do |ssh| hostname = ssh.exec!("hostname") puts "Connected To #{hostname}" puts "Searching for Segmentation Faults" ssh.exec!("sudo tail -n100 /var/log/httpd/error_log | grep 'Segmentation'") do |channel, stream, data| if stream == :stdout puts data.split("\n").first end end puts "Disconnecting From #{hostname}" puts "--------------------------------" end end
Obviously, it’s dead simple, basically connects to an arbitrary amount of servers and tail the last 100 lines from the apache error log, and spit out any that have ‘Segmentation’ in them! No more manual searching of log files!
I think the next thing on the list is to have it graceful the apache server automatically on finding a segfault.
