iOS SDK URL Scheme Competition

iOS has a great “custom URL” feature. It allows apps to be launched from Safari and other Apps.

However, if you have multiple apps with the same URL scheme, you will run into trouble. The following is the result of my testing in iOS 4.2 on the topic (I believe it holds for 3.x too, as I recall testing it back then).

If you install multiple apps which advertise the same custom URL scheme, basically the first one to be installed wins (i.e. is the one that is opened when the link is opened through iOS).  If the currently registered app for an url scheme is removed, then no other apps will respond to that url scheme until they are updated or installed.  So basically, iOS only updates it’s internal url scheme registry when the program is updated/installed.

This means, it is possible that no application will respond to your custom url scheme, even if an app has been installed that supports it (until one of them is updated).  Fortunately you can test whether or not your own app will respond to it’s registered url schemes using [[UIApplication sharedApplication] canOpenURL:.  In the event that the url scheme won’t work due to the app being deleted, and no other apps being installed, that will return false.

The way around this mess is to have a unique url scheme per App, and possible with a common one that they all fight for. So if you know the specific one, you can call it – but if you don’t, you can use the generic one.

Having a unique url-scheme per app is also a useful way to determine if that app is installed or not (don’t use a shared url scheme for this due to the stated problems).

Using the two techniques (the first to determine if any other apps are installed, and the second to determine if your own app still responds to its url scheme), it is possible to develop a workaround to the Facebook SSO issue where multiple apps competing for the same fb- url scheme have issues. Basically:  advertise the fb url scheme on all your apps.  Make sure all your apps have their own unique url scheme.  Then, first test whether or not iOS can actually respond to the fb url scheme (to cater for the case where the app that did was deleted). Second, check all your apps [ones competing for the same fb URL scheme] to see if they are installed – and only if they are not, allow the Facebook SDK to use SSO.  Otherwise, use the fallback of the internal dialog.

Sina’s API is Down

Sina (China’s Twitter) has a public API like twitter does, and GPS Log integrates with it.

Sadly the API is down for maintenance until Monday (from at least 2 days ago).  Terrible!

{“Request”: “/ statuses / upload.json”, “error_code”: “400″, “error”: “40070: Error: limited application access api!”}

Thread 1 (not speaking chinese, I had to express myself with emoticons…)

Thread 2

Official response (translated by google): “Sorry, in order to better service for everyone, to strengthen the management of applications, we are upgrading the platform, suspended some interface call is expected to end next Monday, I’m sorry to give us some inconvenience. Please continue to support us.”  (不好意思,为了更好的为大家服务,加强对应用的管理,我们目前对平台进行升级,暂停了部分接口的调用,预计到下周一结束,非常抱歉给大家带来了不便.请大家继续支持我们.)

As far as I can gather, no third party Sina applications can post status updates for at least the last 3 days, and apparently for another 3.  This is quite pathetic.

Climate Science

“Scientists are the greatest sceptics out there. The second you find someone who’s absolutely certain about something – as some of the climate skeptics are – you’ll know you’re not dealing with a scientist of a serious nature.”
– Tim Flannery, on QandA, Episode 03, 2001.

This is a very good point I think. Scientists are trained to challenge everything. I wonder if climate skeptic “scientists” actually challenge their own internal views.

Day of Ruby Bugs

Learned a lot today.

• Created and deployed a migration
• Created my desired JSON format
• Updated rails on dev machine and server.
• Configured geospike.com’s rails server to talk to the PHP update server using JSON over REST

hit a lot of bugs too

1. reload! doesn’t reload the console in 3.0.0 and 3.0.4
2. as_json is not processed correctly in 3.0.0
3. dump format :sql support is not really as seemless as it sounds

I used sudo gem update rails to update rails, and got an error needed to be worked around (by going to directory cd /Library/Ruby/Gems/1.8/gems/rails-3.0.4/ and creating the ‘lib’ directory). Probably better to use bundle update rails instead…

I learnt a bit about Ruby hashes (aka hash tables, dictionaries). Including that in the latest version of ruby, the hash-rocket syntax has been replaced with the much nicer colon.

Believe it or not, I had not added any class members before today, since rails does this for you from your DB schema.

Rails has some useful from_json and to_json functionality, but they assume model objects. If you just want to parse from JSON into a plain hash, you can use the JSON gem.  Yahoo has a decent example of making a HTTP call to a REST API in ruby, and the relevant API docs are useful too.

If you want to share code between controllers, helpers are not the way, as rails designates them for Views. You could create stock standard modules, but since everything inherits from ApplicationController, I think that’s not a bad place to put stuff.

Appending objects to arrays is easy, with the << operator.

Updating rails on the server is straight forward with capistrano, as long as you require "bundler/capistrano" in your deploy.rb. Simply:

#from the project directory...
bundle update rails
# now edit the projects Bundle file and update the referenced version of rails
bundle package # package up the updates for version control. ref: http://www.ruby-forum.com/topic/216091
git commit -a  # commit  the package
cap deploy # since you committed the bundle the gems will now be available on the server
cap deploy:migrate # if you have any db changes...

The jury’s out on our form views (which are very incomplete). But some tips are here, for future reference.

An interesting day of learning. Those API bugs stumped me for a bit – as being new to rails, I often first assume that the mistake is my own. But they are nothing in the face of just how powerful Rails is.  One great example is how capistrano behaved when I hadn’t done the bundle update and commit.  I was deploying to staging – so no worries even if it did break – but it basically realised that Rails 3.0.4 wasn’t available on the server (specifically, not available in my bundle – as bundled gems are no longer server-wide, which itself is awesome) and rolled back the deploy.  Given that my method of deploying PHP was (and still is) using FTP uploads… this level of automation and environment sanitiy checking is unreal.  In fact, the last PHP project I deployed was developed on a new version of PHP but running on a slightly older one (thanks to RHEL…). Rather than gracefully telling me that the environment wasn’t correct (as per rails with capistrano), I had to execute every line of code to see which ones failed…

Objective-C and Invalid UTF-8 Byte Sequences

I found out the following information the hard way. I hope it helps you in some way…

Objective-C does not handle invalid UTF-8 byte sequences gracefully.

NSString’s UTF8String method will simply return nil if it encounters any invalid chars (which could happen when deserializing a slightly corrupted file from disk, for example). The encoding conversion methods, even the ones that allow “lossy” conversions, will either return nil – or only return the first part of the string up to the invalid sequence. I guess “lossy” here only refers to valid characters.

I had a fiendish problem with one user, where a presumably corrupt char in the DB would cause UTF8String to return nil, and the whole thing fell over. I tried many alternatives to UTF8String like  NSData’s dataUsingEncoding:allowLossyConversion, and NSString’s getBytes:maxLength:usedLength:encoding:options:range:remainingRange.  The former simply returned nil too, the latter would give me the first part of the string.

What was worse, is that [stringToCheck canBeConvertedToEncoding:NSUTF8StringEncoding] would actually return YES (but then not convert).

My conclusion is that all these methods, and everywhere that “lossy” is referenced is designed for converting valid strings from one encoding to another. What I had was an invalid NSString, which these methods are not designed for, even canBeConvertedToEncoding:NSUTF8StringEncoding.

Third Party Solutions Are Hard to Come By

I searched outside the Objective-C/iOS SDK, and found a few leads. One was this solution which uses the Omni framework. I didn’t try it, as I didn’t want to add a massive dependancy for such a simple, rarely used function. It seems that people commonly use GNU’s iconv for the task, e.g. in Ruby and PHP, but I have no idea if I can even compile that for iOS, not to mention the whole LGPL issue.

My Solution to Fix Corrupt NSStrings

In the end, my solution was this: for strings that fail the UTF8String conversion (so only in 0.01% of the time), get every single UTF16 char from the NSString, and make it’s own NSString. Then, run UTF8String on just that single-char string to test if it worked, and ignore those that fail (actually, I replace them with the replacement char: ‘�’) . It’s not pretty, it’s not fast – but it works. And since this is a very rare case, generally the only performance hit you get is the overhead of the UTF8String test.

The only way I know how to test for sure that you can convert to UTF8 is to actually do the conversion via NSString’s UTF8String. It’s heavy, but it’s accurate…

Here’s the method:

/* UTF8 fixup methods, by William Denniss, http://williamdeniss.com/ */

/*
 * Convenience method to do the check and validation in one.
 */
+ (NSString*) makeValidUTF8:(NSString*) stringToCheck
{
	if (![Util isValidUTF8:stringToCheck])
	{
		return [Util removeInvalidCharsFromString:stringToCheck];
	}
	else
	{
		return stringToCheck;
	}
}

/*
 * Returns true if the string can be converted to UTF8
 */
+ (NSString*) isValidUTF8:(NSString*) stringToCheck
{
	return ([stringToCheck UTF8String] != nil);
}

/*
 * Removes invalid UTF8 chars from the NSString
 * This method is slow, so only run it on strings that fail the +Util::isValidUTF8 check.
 */
+ (NSString*) removeInvalidCharsFromString:(NSString*) stringToCheck
{
	NSMutableString* fixedUp = [[[NSMutableString alloc] initWithCapacity:[stringToCheck length]] autorelease];

	// iterates all characters of the string to check
	for (NSUInteger i = 0; i < [stringToCheck length]; i++)
	{
		// gets the character as a one-char string
		unichar character = [stringToCheck characterAtIndex:i];
		NSString* charString = [[NSString alloc] initWithCharacters:&character length:1];

		// converts it individually to UTF8, testing for errors
		if ([charString UTF8String] == nil)
		{
			NSLog(@"Invalid UTF-8 sequence encountered at position %lu. Code: %hu (%X). Replacing with \ufffd", (unsigned long) i, character, character);
			[fixedUp appendString:@"\ufffd"];
		}
		else
		{
			[fixedUp appendString:charString];
		}
		[charString release];
	}

	NSLog(@"Util:makeValidUTF8 WARNING: string was NOT valid utf-8.  Orig length %d, fixed length %d", [stringToCheck length], [fixedUp length]);

	//NSAssert([fixedUp UTF8String] != nil, @"still nil");

	return fixedUp;
}

Incidentally, the character that was in my NSString which caused UTF8String to fail and caused all my grief was U+D843

GPL Stupidity – MobileVLC removed from App Store

MobileVLC (VLC for iOS) is one of the best apps ever for iOS.

It’s not that it delivers on a new use-case (as say I hope does GPS Log) – it’s that it fills a massive hole in Apple’s platform in that it lets you play video files. All of them.

No more did we need to:

1. find a conversion tool.
2. convert the video file (which can take about half the length of the movie)
3. drag it into iTunes
4. Sync
5. Only to find out a few hours later on the plane that the conversion went bad, and all you have is an hour of green.

Instead you just drag the file into the VLC app on iTunes. Then watch it. Wow…

This is great on the iPhone, but for the iPad, it basically makes the device (I don’t believe playing a movie on an iPad should be any harder than a computer).

The best part was – Apple didn’t mind.

Yep – they approved the app, and left it there untouched.  I downloaded it quickly, assuming that Apple may reverse their decision as they do so many times. But they didn’t. Maybe they too realise just how awesome this makes iOS.

So why has VLC been removed from the App Store? Would you believe – due to a complaint by one of the contributors to the parent VLC project.  His name is Rémi Denis-Courmont, herein referred to as dimwit.

Apparnetly dimwit claims that the AppStore licence is not totally compatible with the GPLv2.

Here’s something: who the fuck cares?

This story is not about some evil company who takes GPL code and uses it in their proprietary products without caring about releasing the source.

No, this about some hard working blokes at Applidium who have ported VLC to iOS – and released the source code, as per the GPL.  Hell they didn’t even charge any money for you to download it via the App Store (something that is technically legal under the GPL – but something that I could understand contributing upstream developers may dislike).

Applidium took a GPL project. They ported it to a new platform, released it for free, and released the source. They did this with the permission of the maintainers of VLC.

Maybe – somehow –  this violates the letter of the GPL, but how the hell does it violate the spirit of it?

In the past I have been a huge supporter of the GPL and it’s defenders – but this is a case that goes too far, and to the detriment of users everywhere.  Perhaps some people in the GPL sphere dislike Apple’s “walled garden” approach to their platform, and their efforts to restrict user’s freedoms. This is a very valid criticism – but preventing people from distributing GPL software on the platform is ridiculous, and itself restricting the freedoms of users.

Well anyway – being GPL software, you can still download and compile MobileVLC yourself.  I have – and it works great. If you are a friend of mine, I can possibly provision you a copy.  To compile it yourself, you need to be a paid up iOS developer with Apple, and not be scared by compiler errors.

</rant>

Building VLC for iOS

How I built VLC for iOS today. If you like, you can read my rant about the politics of Mobile VLC.

YMMV – this is a very active project it seems. The steps are probably the same, but your error resolution may be different. If you don’t care about getting the latest and greatest, I *could* zip up my build directory and throw it online, in case that would help.

1. git clone git://git.videolan.org/MobileVLC.git
2. ./buildMobileVLC.sh

In doing this, I got the error (YMMV, this may be fixed):

[info] Building MobileVLCKit (MobileVLCKit)

** BUILD FAILED **
The following build commands failed:
MobileVLCKit:
	Libtool /Users/will/iphone/OSS/MobileVLC/ImportedSources/vlc/projects/macosx/framework/build/Release-iphoneos/libMobileVLCKit.a normal armv7
(1 failure)

Running in verbose mode reveals the actual line - it can’t find the xtag.a library.

Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/libtool: can't open file: MobileVLC/ImportedSources/vlc/projects/macosx/framework/../../../install-ios-Simulator/lib/vlc/plugins/misc/libxtag_plugin.a

[edited for brevity]

Don’t bother trying to open the library xcode projects – if that shell script doesn’t finish, it won’t work.

This failure was due to it trying to link in the “xtag” library, which apparently is no longer needed.

To fix, edit the file:

MobileVLC/ImportedSources/vlc/projects/macosx/framework/AggregateStaticPlugins.h

and remove the line:

plugins+="misc/xtag "

3. Finally, from the MobileVLC project, build the project and run

This blog details a previous successful attempt at getting this compiled, I didn’t actually use these steps since it seems the source is now different from then – but there it is anyway.

Good luck with VLC – if you’re an iOS developer, why not provision a few friends and give them a private copy of VLC.

After all, VLC really *makes* iOS devices…