Compulsory Cron Addons

Cron is a great tool. But it has some shortfalls (some of which I would probably put in the ‘defect’ basket).

1. Cron emails you unless the stdout and stderr output is totally blank

Problem is that you have to have very very well behaved scripts not to have cron email you. But if they are too quiet then maybe when you get a problem useful debugging info is missed.

Solution: Cronic.

This neat little shell script solves Cron’s “cronic” email problems.

This little script will only output messages if the error code is non-zero. Added the nessesary lines to my inner scripts so that bash will return error codes if any of the called programs do.

neat. get cronic.

2. Your .bashrc environment isn’t loaded when a cron task runs

This isn’t a defect per-say but a catch that you’ll need to workaround if you rely on the goodness of your .bashrc file, e.g. if your ruby install is provided by rvm.

two good options for that are here.

Summary:

# in crontab
* * * * * /bin/bash -l -c '/path/to/script'

# or, in the script
source ~/.bashrc

3. A cron task can run overtime and a duplicate started

The problem is that sometimes a cron job (especially an every 5 minute one!) can run over time. Cron faithfully starts a new task, further loading the CPU, this repeats for a while until your system grindes to a halt (this happened to me recently).

The solution is to use lockrun, a great little tool that you can easily compile yourself, to prevent cron starting the same task twice.

Important note: when specifying the lock file don’t use the ~yourusername/ path shortcut. Make the path the full /home/yourusername, as it seems that lockrun doesn’t support the shorthand way.

Conclusion

So finally, at the end of the day my crontab looks a little complicated, but I pipe through cronic, then lockrun. The bash env stuff I setup in the script itself.

Disqus on pre-JSON supported versions of PHP doesn’t support Unicode

Disqus’ API supports Unicode nicely.

If you’re using an old version of PHP however (before json_decode was added), then you won’t be getting this support using the Disqus PHP client libraries. That is because the json library bundled with Disqus as a fallback for older PHP versions doesn’t support unicode at all (it simply strips the \ character leaving you a bunch of crap like u064au0645 u064au0645 u0648u0648u0648u0627u0648u0648u0648.

You have two choices - upgrade PHP, or replace Disqus’ fallback json library with something that works. I switched out the json library for one I use elsewhere on my site, and it works great now.

Mysql::ProtocolError with ruby-mysql

Was getting a “Mysql::ProtocolError” when running the ruby-mysql gem with the 64bit version of MySQL on Mac OS X 10.6.

Stackoverflow said to downgrade the version (done – modified the Gemfile to specify the exact version), but I still had to modify one line of code to get it to work.

The line of code that needed to be removed is this one

To find where that file is on your system, type gem env. In my case it was cd /Library/Ruby/Gems/1.8. Then navigate to the file in question, for me this was vi gems/ruby-mysql-2.9.3/lib/mysql/protocol.rb +666.

The issue for ruby-mysql has been raised here.

Aside: why are there so many mysql<->rails issues? The native gem is a bitch to compile, the ruby one has this issue, and we are using a different gem in development on Mac to production on Linux all to workaround various issues.

Setting a branch to be that of another branch in git

Sometimes you start working in a branch, and that branch basically becomes the master. I had a problem once where merging this branch didn’t go so well – and what was the point anyway? That branch had become my new master.

The solution is the ours merge strategy (not to be confused with the ours-conflict resolution option). Thanks StackOverflow

Q: Which to implement – RSS or ATOM or Both?

A: If you’re a content-publisher: it does not matter in the slightest. Just pick one that you like. In fact, I think it’s a mistake to implement both. Read on.

There are two main “feed” formats. But here’s the thing – publishers don’t need to support both, readers do. If you’re writing a feed-reader, then sure, you have to support both. But as a content-publisher what’s the point? All readers worth their salt will support both. So just pick the format that you feel will better describe your data and use that one.

There are actually disadvantages in supporting both as a publisher, and that is if you advertise multiple feeds on the same page (e.g. one for RSS and one for ATOM, even though they are identical), the user will often be prompted as to which one they want (see screenshot). This forces the user to choose between two identical outcomes. So what’s the point? I don’t want my Mum asking me which one she should use.

FWIW I favour Atom, I think it’s better formed.

My WordPress theme (and I’m guessing others too) by default advertised 3 feeds. This post was actually sparked by my work onGeospike, not this blog – but I will eat my own medicine here as well. Hence, OmegaDelta now only advertises ATOM. The old feeds will still work, for clients that may have subscribed to them in the past.

On the topic of ATOM feeds, make sure you validate, to improve your chances of the feed working in readers (hint: if it’s not valid, forget about it!)

Man up and install MediaWiki in the Root Directory

[Editors note: items italicised in this post are meant to be read in a sarcastic ghooostly like tone, the reader is welcome to replace 'Men' for 'Women' and 'Boys' for 'Girls' in their own minds as the read this, the intent is not to be sexist]

Apparently installing MediaWiki in the root directory is bad, and if you do this you will not have any support. Two of the arguments seem to be “if you do this then you’ll have to set up special rules for robots.txt and favicon.ico”.

So. What?

Link squatting (i.e. using reserved locations, not to be confused with domain squatting) is a bad, non-W3c recommended practice. So new such schemes should hopefully be non-existant.  And, as it pointed out, it is possible to create custom rules.

They recommend you read ‘Cool URLs don’t change’ before jumping “willy nilly” into using the root directory.  Fine. But cool URLs that don’t change *are* ones without needless cruft in the url like “wiki/”.  A fact that becomes especially relevant when you are already using a subdirectory for the wiki.  http://wiki.example.com/Foo is cool.  http://wiki.example.com/wiki/Foo is a bit redundant isn’t it?  (as to be frank is http://en.wikipedia.org/wiki/Main_Page).

Does wikipedia.org host other non-wiki related content? An email service perhaps at wikipedia.org/email?  No, of course not.   We’re talking about dedicated wiki domains/subdomains here – why add cruft to the path?

After all – at http://Geospike.com people can create their own usernames on the root directory. Does this present challenges?  Sure. So what?  We’re men, not boys.

Since I’m ranting about this, I’m always amused to see link shorteners that add cruft to the urls like a “/p/” prefix, or “-” prefix.  Kinda defeats the point of link shortening doesn’t it? Why not do the opposite?  Say all links must be > 4 characters, then put any adminy stuff in directories with < 4 chars. Seems easy to me.  I’m just saying…

As far as media wiki is concerned, man up and just install it where you like. I used the following .htaccess file to set it up, and redirect the old links, based on this info.

RewriteEngine On

RewriteRule ^/?page/(.*)$ http://wiki.gpslog.cc/$1 [L,R=301]

RewriteRule ^[^:]*\. - [L]
RewriteRule ^[^:]*\/ - [L]
RewriteRule ^/*$ /index.php/Main_Page [L,QSA]
RewriteRule ^(.+)$ /index.php/=$1 [L,QSA]

Moving my iTunes Library for the last time

Again I find myself with a big iTunes mess. The thing is, I don’t want to use iTune’s “consolidate library” feature, I have my music nicely organised thank you.  I keep my music on an external hard drive, and over the years this can change.

Moving this directory is a real pain, because iTunes doesn’t recognise the move, nor does it have a batch “change the base path” feature (which is sorely needed).

The smartest thing you can do is create a symlink (hint: ln -s in the console) in your iTunes directory that points to your external HDD.  This way, the next time you do a move – all you have to do is modify the symlink!  Even if you move the files back to your main HDD…    so this is what I did (and now I’ll never have to do what I did next again…)

Alas I had to fix some paths from before when I started using this symlink method.  What follows is a summary on how to update the path for 1000s of songs in iTunes (pointing to my new shiny symlink, which in turn points to the external HDD):

NOTE: this is simply what I did.  I don’t recommend doing or not doing anything in this post, use at own risk!

[Apple's Docs on this are here]

1. Close iTunes
2. Browse to your iTunes folder.  Backup the files “iTunes Library” and “iTunes Music Library.xml”.  (restore these if anything blows up).
3. Make another copy of the .xml file somewhere handy, like your desktop
4. Edit this XML file.  Do a find and replace for the directory string you need to replace (hint: to validate the file:// URLs you are using, just copy+paste one into Chrome, if it finds it, you’re all good best to do this to avoid errors).
5. Now open iTunes, and remove all the music and playlists (you can always restore the backup in #2)
6. Then File -> Library -> Import Playlist.  Select the XML file you edited on your desktop.
7. Wait.  This takes ages.

And finally, all my music is back.  By using the symlink trick, I can now move this directory around to my hearts content.  Yey.

Important Note: if you do setup the symlink then make sure when you add new music you drag it from the symlink directory!

Testing in-app purchases in iOS 4.3 sandbox

In-app purchases have always been a bit painful to test. But recently (with iOS 4.0+), I’ve had a terrible time trying to get my iPhone to use the sandbox for testing. The problem is it often thinks it’s in the production environment *even* if you enter the in-app purchase test user (the symptom of this is when you are asked to confirm all sorts of details like the credit card number for the test user… that’s bad!).

I also had a problem where even after I had signed out of my AppleID on the iPhone it would still pop up with that AppleID asking for the password.

Here’s how I fixed it. Doing these steps seem to finally get it to use the sandbox (it’s possible some may be unnecessary – but when I do them all, it works):

1. Sign out of your account on the iPhone in Settings -> Store
2. Delete your test app
3. Power the iPhone off and on again
4. Close iTunes on your computer
5. Run the app again from XCode

YMMV…

How to wait for iOS methods with completion-blocks to finish

iOS completion blocks rule. A little bit of code that runs after a (normally slow) task finishes, with full access to the classes member variables, and even read-only access to the enclosing method’s local variables.

However, sometimes you want these calls to be blocking.

*shock horror* you may be thinking. Why hasn’t he re-designed his app flow to take advantage of these new methods? (this was the general response that one guy got to his question “How to wait for methods with result/completion blocks to finish?“).

Well the answer to that question is that I already designed my UI to be fast and responsive, with cancellable actions – before it was cool I guess. So all my code is already run in worker threads. So now when I want to implement something like ALAssetManager image loading, I want to add it to code that is already nicely multi-threaded, and designed to support other methods of loading images (that use blocking reads, rather than the newer completion block design). I’d rather not totally mess up the flow of my logic, and since it’s already threaded, it’s perfectly fine to just block and wait for the asset manager to return.

So how do you do this?

The answer is NSLock. Specifically NSConditionLock. Basically you create a lock with the condition “pending tasks”. Then in the completion and error blocks of assetForURL, you unlock it with the “all complete” condition. With this in place, after you call assetForURL, simply call lockWhenCondition using your “all complete” identifier, and you’re done (it will wait until the blocks set that condition)!

A big caveat that applies to ASAssetManager (but not always other iOS APIs using a similar block-callback structure) is that runs some code on the main thread. So to use this system you *must* call it from a thread. As I said in my pre-amble, the reason I want to block is that this is already in a worker thread anyway, so it shouldn’t be a problem (if it is for you, then reconsider your design – it’s not a good idea to block up the main thread).

Here’s some rough code as an example. I haven’t made it into a utility for you to use, as you probably want to understand what’s going on here, rather than dragging and dropping my code into your project.

// --------------- .h file

	// class members in the header file (can't be local as then the blocks wouldn't be able to use them

	NSConditionLock* albumReadLock;
	NSData* defaultRepresentationData;

// --------------- .m file

// NSConditionLock values
enum {
    WDASSETURL_PENDINGREADS = 1,
    WDASSETURL_ALLFINISHED = 0
};

// loads the data for the default representation from the ALAssetLibrary
- (NSData) loadDataForDefaultRepresentation:(NSURL*)assetURL
{
	// this method *cannot* be called on the main thread as ALAssetLibrary needs to run some code on the main thread and this will deadlock your app if you block the main thread...
	// don't ignore this assert!!
	NSAssert(![NSThread isMainThread], @"can't be called on the main thread due to ALAssetLibrary limitations");		

	// sets up a condition lock with "pending reads"
	albumReadLock = [[NSConditionLock alloc] initWithCondition:WDASSETURL_PENDINGREADS];

	// the result block
    ALAssetsLibraryAssetForURLResultBlock resultblock = ^(ALAsset *myasset)
    {
        ALAssetRepresentation *rep = [myasset defaultRepresentation];

		NSLog(@"GOT ASSET, File size: %f", [rep size] / (1024.0f*1024.0f)); 

		uint8_t* buffer = malloc([rep size]);

		NSError* error = NULL;
		NSUInteger bytes = [rep getBytes:buffer fromOffset:0 length:[rep size] error:&error];

		if (bytes == [rep size])
		{
			NSLog(@"Asset %@ loaded from Asset Library OK", self.assetURL);
			defaultRepresentationData = [[NSData dataWithBytes:buffer length:bytes] retain];
		}
		else
		{
			NSLog(@"Error '%@' reading bytes from asset: '%@'", [error localizedDescription], self.assetURL);
		}

		free(buffer);

		// notifies the lock that "all tasks are finished"
		[albumReadLock lock];
		[albumReadLock unlockWithCondition:WDASSETURL_ALLFINISHED];
    };

    //
    ALAssetsLibraryAccessFailureBlock failureblock  = ^(NSError *myerror)
    {
		NSLog(@"NOT GOT ASSET"); 

        NSLog(@"Error '%@' getting asset from library", [myerror localizedDescription]);

		// important: notifies lock that "all tasks finished" (even though they failed)
		[albumReadLock lock];
		[albumReadLock unlockWithCondition:WDASSETURL_ALLFINISHED];
    };

	// schedules the asset read
	ALAssetsLibrary* assetslibrary = [[[ALAssetsLibrary alloc] init] autorelease];

	[assetslibrary assetForURL:assetURL
				   resultBlock:resultblock
				  failureBlock:failureblock];

	// non-busy wait for the asset read to finish (specifically until the condition is "all finished")
	[albumReadLock lockWhenCondition:WDASSETURL_ALLFINISHED];
	[albumReadLock unlock];

	// cleanup
	[albumReadLock release];
	albumReadLock = nil;

	// returns the image data, or nil if it failed
	return defaultRepresentationData;
}

Installing OS X from a Flash Drive

Pretty neat if your DVD drive is flakey, you’ve lost the installation DVD, etc.

Here’s how.

Basically you format the Flash Drive to be bootable, then “restore” the DMG image of OS X onto it. Neat.