Fixing POST errors that seem almost random

Recently my webhost installed mod_security on my webserver. It seemed like a non-event until I started getting calls from a few clients who had been merrily working for a long time. When they tried to update content in either Open-cart or WordPress, they were getting strange errors such as “501 Not Implemented” or “406 Not acceptable”.

The fix is to sidestep mod_security for those requests. Ideally it should be limited to the offending request, but in the interest of getting people running, I simply added a .htaccess file in the /admin folder of the broken sites with the following contents.

[code]

SecFilterEngine Off
SecFilterScanPOST Off

[/code]

Installing a 3.0 kernel on Ubuntu 11.04 (natty)

Ok, I saw the warning but didn’t think it applied to me. Don’t assume it doesn’t apply to you!

Remove any proprietary ATI/AMD drivers from your Natty installation. You can put them back on after we install the new kernel.

Now that you’ve checked for anf removed any ATI proprietary drivers, visit this URL and pick out the kernel that you want to try. http://kernel.ubuntu.com/~kernel-ppa/mainline/

In my case, I was running the latest natty kernel on a new (but low-end) Toshiba C650D laptop. It was crashing constantly, never running for more than an hour. As there weren’t any clear indicators in the logs, and the hardware worked fine on Windows, I wanted to try a different kernel. I wanted to the newest kernel possible in the hopes that whatever issue had been resolved.

I opened the above Url and scrolled way down to the bottom of the page. All the way to “v3.0.4-oneiric”. It was dated four days ago – that ought to be new enough! As I’m running 64 bit 11.04, I need the “amd64” binaries. Download the appropriate ones, “headers…all” and the “headers” and “image” file for my architecture.

[code] linux-headers-3.0.4-030004_3.0.4-030004.201108301138_all.deb linux-headers-3.0.4-030004-generic_3.0.4-030004.201108301138_amd64.deb linux-image-3.0.4-030004-generic_3.0.4-030004.201108301138_amd64.deb [/code]

Next, open a terminal window and cd to the folder where your binaries are located. Run the following command to install the new kernel: [code] /sudo dpkg -i /linux-headers-3.0.4-030004_3.0.4-030004.201108301138_all.deb linux-headers-3.0.4-030004-generic_3.0.4-030004.201108301138_amd64.deb linux-image-3.0.4-030004-generic_3.0.4-030004.201108301138_amd64.deb [/code]

Let it run for a minute or two and if there are no errors reboot. If you have errors, fix them, try the install command again and reboot. If Grub doesn’t prompt you to choose a kernel, check it from terminal once you’re logged in. with the following: [code] /uname -r /[/code]

There you have it! Running a 3.0 kernel on Ubuntu 11.04, Natty.

Domain Registration – on my schedule!

Working as a freelance web developer means that I work really strange hours. I may not get to a project until my kids are in bed and keep working late into the evening. When there’s something that needs to be done, nothing ticks me off quite like a broken website that requires a phone call to customer support. Once or twice I’ll let someone get away with it, after-all sometimes stuff breaks. That’s just the way it is. But when it came time to renew my website domain – the face of my freelancing business, and I discovered yet again that the domain renewal pages of my registrar *still* would not allow me to renew online I nearly lost it. I called up their customer support where the person on the other end asked me for my username and password so that they could enter the transaction for me. Yeah right. It was time to find myself a new registrar. After the usual poking around the Internet and some discussion among my associates, I settled on http://www.namespro.ca/. They’ve been in the business since 2003 and are a Canadian business so I feel like I can trust them to manage my domains properly. I signed up and was able to quickly and easily move over my domain. The website is easy to follow and the domain management pages are clean and easy to use. I sent a couple questions out to their tech support and received prompt replies that made me very happy with my choice. I’ve since moved a couple more domains to them and will continue to chance my domains over to namespro as they come due. Managing my domains from a proper web interface on my own schedule has never been simpler!

Resize and Rotate

Last week my laptop died so when the new one arrived I started re-installing all those apps that make it work the way I want. This post is about one of those *must-have* applications. I manage all may camera images on my laptop so being able to quickly resize or rotate images is pretty important to me. Luckily a friend showed me an easy way to do bulk rename and resize with a simple right-click.

Installation

To add the new options to the Nautilus context menu we’ll first need to install the package:

[code] sudo apt-get install nautilus-image-converter [/code]

You’ll need to restart Nautilus or simply logout and log back in before you get the new right-click menu options. After you’ve done that you will be able to right-click on any image file and you’ll see the two new menu items: “Resize Images” and “Rotate Images”.

When you choose one of the menu items you will be presented with a simple interface to provide the relevant input parameters. What could be easier?!

Embedding subtitles in an AVI with mencoder on Ubuntu

I should point out that I didn’t write this script, only that I found it useful while trying to put a movie with subtitles in a separate file (*.srt) onto my Android phone.

Here’s where I found the original script: http://ubuntuforums.org/showpost.php?p=7809155&postcount=6

I put the following script into a text file named hardsub.sh and copied it into “~/.gnome2/nautilus-scripts”. Then restarted Nautilus and copied the (movie).avi and (movie).srt files into the same directory and ran it by right-clicking on the video and selecting “scripts -> hardsub.sh”. It ran for a while and spat out a file named (movie).hardsub.avi that included the embedded subtitles.

Here’s the script: [code] #!/bin/bash

# hardsub.sh # a basic script to hardcode subtitles into movie files using mencoder # # uses a two pass – high quality encode by default # fast option invokes a quick one pass encode # # it assumes that you are using an .srt subtitle file with the same #+ name as the movie file # # if you are using a matroska movie (.mkv) and it can’t find the .srt #+ file it uses the embedded english subtitle track # # # NB: this is a preliminary script and doesn’t do much error checking # # Version 0.11 # # Mike Lahey 2009 #####################

# language used for subtitle track embedded in a matroska file # for alternative subtitle (& audio) tracks try “mplayer -v movie.mkv” LANG=eng

# these mencoder options scale down the subtitles and move them #+ up just a bit SUBOPTS=”-subfont-text-scale 3.3 -subpos 96″

if [ -z “$1” ]; then echo usage: $0 movie.avi [fast] echo echo fast – do a fast one pass encode exit 1 fi MOVIE=$1 EXT=${MOVIE##*.} OUTPUT=${MOVIE/%.$EXT/.hardsub.avi}

# first make sure that our movie file is there if [ ! -f “$MOVIE” ]; then echo input file “$MOVIE” does not exit exit 1 fi

# next check if we have the SRT file and create subtitle command for # mencoder SRT=${MOVIE/%$EXT/srt}

if [ -f “$SRT” ]; then

# good we have an SRT file, so create our subtitle command SUBCMD=”-sub “$SRT”” else #no SRT file? well check if we are dealing with a matroska file if [ “$EXT” == “mkv” ]; then echo “################################################” echo couldnt find “$SRT” echo so trying embedded subtitle track “$LANG” echo “################################################” SUBCMD=”-slang $LANG -spualign 2″ # “-spualign 2” ensures proper placement of vobsub tracks else echo could not find subtitle file “$SRT” echo echo maybe its called something else? try renameing it. exit 1 fi fi

# if the input movie is not an avi then transcode the audio to mp3 # this may result in an unnecessary transcode but it will make #+ incompatable audio streams (such as ogg/vorbis) work with an avi

if [ “$EXT” == “avi” ]; then ACODEC=copy else ACODEC=mp3lame fi

# most formats other than avi allow non-square pixels. # by invoking the scale filter we ensure that the video is # displayed with the proper aspect ratio in our avi # since were scaling anyway – scale to a multiple of 16 to make the encoding more effiecient if [ “$EXT” != “avi” ]; then SUBOPTS=”$SUBOPTS -vf-pre scale=-8:-8″ fi

#### if fast option was invoked, do a fast one pass encode if [ “$2” == “fast” ]; then

CMD=”mencoder “$MOVIE” $SUBCMD $SUBOPTS -o “$OUTPUT” -oac $ACODEC -ovc lavc” echo $CMD eval $CMD

if [ $? != 0 ]; then echo “################# mencoder failed ################” exit 1 fi exit fi

#### normal two pass encode

CMD=”mencoder “$MOVIE” $SUBCMD $SUBOPTS -o /dev/null -oac $ACODEC -ovc lavc -lavcopts vcodec=mpeg4:vhq:turbo:vpass=1″ echo $CMD eval $CMD

if [ $? != 0 ]; then echo “################# mencoder failed ################” rm divx2pass.log exit 1 fi

CMD=”mencoder “$MOVIE” $SUBCMD $SUBOPTS -o “$OUTPUT” -oac $ACODEC -ovc lavc -lavcopts vcodec=mpeg4:vhq:vpass=2″ echo $CMD eval $CMD

if [ $? != 0 ]; then echo “################# mencoder failed ################” rm divx2pass.log exit 1 fi

rm divx2pass.log

[/code]

Using Cron jobs from CPanel

I’ve set this up before but couldn’t remember just how I did it. Instead of having to figure it out all over again next time, I thought I’d write it somewhere.

Setting the time in CPanel’s Standard Cron UI is very simple so I won’t bother with that but the command for triggering a web page has a few things that messed me up.

For those who don’t care to read much, here’s the command:

[code] wget -O – -q -t 1 ‘http://www.mydomain.net/doit.php?id=12345’ >/dev/null 2>&1 [/code]

Or, if you’re using a .htaccess username and password, like this:

[code]
wget -O – -q -t 1 ‘http://username:password@www.mydomain.net/doit.php?id=12345’ >/dev/null 2>&1
[/code]

Of course you’ll need to insert your own url but you can otherwise cut and paste the line above and start running your php script as a cpanel cron job. But what does it all mean?

“wget” is a linux command line utility to fetch the contents of a web page. In our case we don’t care about the output of the page, only that it gets triggered – or ‘looked’ at.

“-O -” says to discard the output (aka, the page contents), we’re not interested to keep these!

“-q” put wget into quiet mode, I don’t want to know about errors here either.

“-t 1” tells wget to only try once, if the page doesn’t work for some reason, give up.

The url should be wrapped in single quotes! This is an easy one to miss because in a simple case, it’s not required but if you have url parameters it’s a must otherwise the url gets truncated and the request won’t work as you planned it to.

“>/dev/null 2>&1” This tells cron *not* to send out a notification that the script was called. The details of this part gets a bit hairy but I’ll do my best to explain. As the script I’m triggering already sends an email when it completes, I don’t really want the one that is sent out automatically each time the cron runs. As well, the subject for the email is the complete cron command. I don’t know about you but if your scheduled cron command includes anything even sort-of private, I’d prefer not to blast it out in email every day! Consider if you included a URL to a script behind an htaccess password, the username and password would be right there in the subject of your email every day!

The first part, “>/dev/null” redirects the output (standard out) of the command to /dev/null (the Linux equivalent of a black hole!). The second part, “2>&1” redirects ‘standard error’ to ‘standard out’. The short story is that there is no more output from your command and therefore there’s no email to be sent out!

Recovering files deleted from an SVN repository

This morning I find myself in need of a file that used to exist in an SVN project. The file was deleted several months ago and I don’t have a copy anywhere else. Finding it in my SVN repository turned out to be quite simple.

Run the following command to dump the complete activity log from the project.
[code]
> svn log –verbose > myLog.txt
[/code]

Then simply open the ‘myLog.txt’ file with your favourite editor and hunt for the last mention of the file you need. I’m looking for “register.module.php”, so for me that looks like this:
[code]
————————————————————————
r428 | mvoorberg | 2010-11-15 22:06:33 -0700 (Mon, 15 Nov 2010) | 2 lines
Changed paths:
M /faq.php
M /login.action.php
D /register.module.php
[/code]

Use the following command to restore the file to the local filesystem at which point you can copy it out or do whatever you like with it. Notice the version number (427) is one less than the version that included the Delete action above.
[code]
> svn up -r 427 register.module.php
[/code]

Once I copied the file out, I removed it again by simply running:
[code]
> svn up
[/code]

Don’t forget to remove the ‘myLog.txt’ file when you’re done with it.

Moving Joomla site to a subfolder

Before rebuilding this website, I wanted to make sure I was able to keep the old site in a subfolder to help keep my search ranking and not break any external links to my site. This is clearly a job for modRewrite so I had a look around and goolged things like “Joomla to subdirectory” and the like. Funny enough I couldn’t find any posts about it and no-one had posted the required .htaccess entries to make it happen. As I’d never used SEF urls with my Joomla site and I wasn’t going to use Joomla for the new website, it was easy to find a pattern that all my urls would match. I could simply send anything that contained the string “option=com” to my “archived” subfolder.

Here’s the entries I added to the .htaccess in the root of my website:
[code]
# Make sure this module is turned on
RewriteEngine on

# Match any Joomla Urls
RewriteCond %{QUERY_STRING} ^option=com(.*)$

#Only forward Get and Head requests to the archived site
RewriteCond %{THE_REQUEST} ^(GET|HEAD)

# Don’t match urls that already contain the archived subfolder
RewriteCond %{REQUEST_URI} !^/archived/$

# Send a 301(Permanent Move) to the archived folder
# and don’t process any more rules(L)
RewriteRule ^index.php$ /archived/index.php [L,R=301]

[/code]

Follow-up:
As it turns out you can’t set the database permissions for specific tables when running in a typical shared LAMP environment. I was hoping for SELECT only and INSERT on some Joomla session tables. I noticed that I was still getting some new spam user registrations on the old site so I’ve disabled POST requests with the following entries added to the .htaccess in the /archive folder of the old Joomla site:
[code]
# Make sure this module is turned on
RewriteEngine on

# Disallow POST requests to the archived site by sending a 404 error
RewriteCond %{THE_REQUEST} !^(GET|HEAD)
RewriteRule ^ / [R=404]

[/code]

C# casting – Did you know?

I was reading through someone else’s C# code the other day when I saw something that wasn’t immediately clear. It went something like this:
[code]
Object o;
for (int i=0; i < myCollection.count; i++) { o = myCollection[i] as SomeObjectType; // Huh? if (o != null) { // Do something with 'o'... break; } } [/code] While the cast seems clear enough, there are a few things to be aware of. If the above cast fails, it assigns a null value to 'o'. If the null value is not handled immediately, you may not realize the problem until some time later when you run into the null reference. [code] Object o; // As-casting - Returns null if the cast fails! o = myCollection[i] as SomeObjectType; // Prefix casting - throws an exception if the cast fails! o = (SomeObjectType)myCollection[i]; [/code] There may be a performance benefit to the As-cast but I'm much more comfortable knowing that my cast exceptions were handled properly using the Java-esque Prefix cast.

Converting RAW images to Jpeg on Ubuntu

You’ll need to install a few packages before this works:
[code]
sudo apt-get install netpbm
sudo apt-get install dcraw
sudo apt-get install libimage-exiftool-perl
[/code]
Once you’ve got these installed, put the following in a file in ~/.gnome2/nautilus-scripts/ and restart Nautilus to add it to the right-click context menu.

[code]
#!/bin/bash
files=$#
count=1
message=`echo “Converting $files RAW files to Jpeg”`

(while [ $# -gt 0 ]; do

# Strip off the file extension, including the “.”
upperExt=`echo $1 | sed ‘s/.*\(\..*\)/\1/’ | tr ‘[a-z]’ ‘[A-Z]’`

if [ -f “$1” ]
then
# Get the file name without the extension
trimmed=`echo $1 | sed ‘s/\(.*\)\..*/\1/’`

if [ $upperExt = “.PPM” -o $upperExt = “.CR2” -o $upperExt = “.NEF” -o $upperExt = “.OTHER_RAW_EXTENSION” ]
then
if [ $upperExt = “.PPM” ]
then
# Convert the PPM image to a Jpeg
pnmtojpeg “$1” > “$trimmed.jpg”
else
# Convert the RAW image to a Jpeg
dcraw -c -w -h -b 1.0 “$1” | pnmtojpeg > “$trimmed.jpg”
fi
# Copy EXIF data to the new Jpeg image
exiftool -overwrite_original -TagsFromFile “$1” “$trimmed.jpg” >/dev/null

# Set the Jpeg’s file timestamp to match the EXIF date
dcraw -z “$trimmed.jpg”
fi
fi
# Output the zenity progress bar
sav=`echo “(($count / $files) * 100)” | bc -l`
echo $sav
count=`expr $count + 1`
shift
done) | zenity –progress –auto-close –auto-kill –text “$message”

[/code]

You’ll have to make the file executable by using:
[code]
chmod +x raw2jpeg.sh
[/code]

Update:
I modified the script to convert PPM files created by UFRaw as well. Also now using pnmtojpeg instead of ppmtojpeg.