Finally, a rough alpha SDK is in the wild without the hindrance of an NDA.

www.isPebbleSDKShipping.com

and its API have been updated accordingly.

Ever seen this gem before popup when working on a Google Web Toolkit project?

"Errors occurred during the build.
 Errors running builder 'JavaScript Validator' on project 'com.your.project'.1"

Here is the trouble. Let’s say you’ve not marked your target/ directory as derived, so that the GWT Eclipse plugin can launch DevMode from there. Since Eclipse can see these files, it will try to validate that compile, minified JS that GWT created and sometimes die. You don’t want this.

So, disable JS validation on your GWT project since the GWT Compiler will take care of that validation for you. Just visit:

Window > Preferences… > JavaScript > Validator > Errors/Warnings and uncheck the box.

Just read Good Relationships by Michael Hunger. It was a great read to get up to speed on Spring Data Neo4J quickly, and also the only reference out there.

But, it requires you join InfoQ and give them your personal information to sell to whoever they please. No thanks.

Login to our evil lair

Well, let’s look at those links, what do they really do? Pop open the inspector and look at the link. We see that it calls loginAndDisplayDownloadLink(). Ok, so what is this function? Well, we can see it displays the content of afterLogin once complete. So, what is in that element? Oh, I see, the download links!

A crappy website, right? Why manage sessions and login state when you can just have a form post away user information from the page and reveal what was always there? Understandably a lower overhead implementation, but not secure by any measure.

So, to get the links, just run this on the page: document.getElementById(‘afterLogin’).style.display = ‘block’

Or use the download links, since the are in a publicly accessible page after all

• PDF (128MB!) – Update: this file appears to be the only secured file
• ePub
• mobi

Of course, this is quite common since MicroSoft Does it Too.

Since learning about the Pebble almost a year ago, I knew it would be awesome.

I’m a runner and maintainer of RunPartner. Stats about runs are important to me. For years I used various Forerunner watches. But, Pebble would be better. I’ve written SDK’s for websites to work with the Garmin watches (github) but I couldn’t change the code on the Forerunner with any neat new ideas. As a software engineer, this is irksome. Yet, the Pebble promises the ability to create my own running watch AND to release it online to anyone to use. Awesome.

After much delay and ballyhooing the watch shipped on January 23rd this year. But where is the SDK? The SDK we were promised. The SDK that I plunked down $150 long ago for because of the myriad promises and claims that it would be available. No. No SDK for you. Well, some SDK for some folks, but an NDA for them.

On the forums much musing is ongoing, but no official missives from the Pebble officials since the single, solitary Dev Mailing List email in October. Radio silence.

Thus, I present the Pebble faithful (it is faith at this point) with an easier way to see if the SDK is out there. An official, unofficial source of SDK release status.

www.isPebbleSDKShipping.com

As anyone using the Google Web Toolkit JSNI knows, there are certain reserved variables, or “dollar words,” exposed by the GWT runtime. While you may be familiar with $wnd and $doc, there are more undocumented words. See the exhaustive (as of GWT 2.5.0) list below taken from source.

Example of accessing GWT Version

To determine the running version of a GWT application, you can use JSNI to return that version.

// easily get the GWT version
public static native String getGwtVersion() /*-{
	return $gwt_version;
}-*/;

From outside the application you can walk the DOM of the page to find the iframe containing the running GWT code and access the version like below.

var gwtVersion = null;
var frames = document.getElementsByTagName('iframe');
for (var i=0; i<frames.length; i++) {
    // prevent security access errors
    try {
        if(frames[i].contentWindow.$gwt_version) {
            gwtVersion = frames[i].contentWindow.$gwt_version;
            break;
        }
    }
    catch(e) {}
}

This is the mechanism I used to detect GWT apps in the Library Detector Chrome Extension (Github). Note the wrapping in a try/catch. This is because some iframes are cross domain and will throw a security exception if you try to access them.

If you’re a pragmatic programmer, then you’ve learned all the shortcuts you can to make your coding day easier. I’ve been a fan of the numerous shortcuts in Eclipse such as Alt+Shift+S,R to create all class accessors/mutators. Still, my favorite has been Ctrl+Alt+Up/Down Arrow and Ctrl+Up/Down Arrow to instantly duplicate a line above or below the current line and to move it around. Who has time to select, copy, move, paste? Plus, you can now save your copy/paste buffer for more important things, like that LinkedIn password you keep changing each time they’re hacked.

But on my new X220 ThinkPad, shortcut just does not work in Eclipse. I was pulling my hair out. Of course I disabled graphics rotation hotkeys from the desktop context menu, but it kept failing to work in Eclipse. WTF? I even double checked the shortcut configurations in Eclipse to ensure it was not changed. No dice.

While pulling my hair out, and killing random processes, I found the SOB. Lenovo, in their great wisdom, has also used the very same key bindings as their graphics driver to rotate the screen in the Lenovo Reading Optimizer. Pull your hair out all night, you’ll just be bald and angry in the morning.

The Fix

1. Disable hotkey rotation on the desktop context menu.

   

2. Uninstall Lenovo Reading Optimizer. 

   (from Add and Remove Programs)

Spring is pretty swell, but the documentation is never what you need. It would be like reading an anatomy book about every organ in the body, but never being shown a picture of all of them together, and then trying to conduct a surgery.

I was assembling Spring Security for a rewrite of RunPartner.com and read the RememberMe chapter of Spring Security.

They first show a trivial case. As is often the case, this is too trivial a case to be of much use.

<http>
    ...
    <remember-me key="myAppKey"/>
</http>

Because I needed a database backed implementation, so I liked the next example.

<bean id="rememberMeFilter" class=
 "org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter">
  <property name="rememberMeServices" ref="rememberMeServices"/>
  <property name="authenticationManager" ref="theAuthenticationManager" />
</bean>

<bean id="rememberMeServices" class=
 "org.springframework.security.web.authentication.rememberme.TokenBasedRememberMeServices">
  <property name="userDetailsService" ref="myUserDetailsService"/>
  <property name="key" value="springRocks"/>
</bean>

<bean id="rememberMeAuthenticationProvider" class=
 "org.springframework.security.authentication.rememberme.RememberMeAuthenticationProvider">
  <property name="key" value="springRocks"/>
</bean>

But there is a lot of cruft there you don’t need, and I wanted a different cookie TTL and cookie name. So I used the below, which seemed to work.

<http>
	...
	<remember-me services-ref="rememberMeServices"/>
</http>
<!-- Handles auto login from remember me token -->
<beans:bean id="rememberMeServices"
	class="org.springframework.security.web.authentication.rememberme.TokenBasedRememberMeServices">
	<beans:property name="userDetailsService" ref="userService" />
	<beans:property name="key" value="secretKey" />
	<beans:property name="cookieName" value="customRememberName" />
	<beans:property name="tokenValiditySeconds" value="604800" />
</beans:bean>

Yet an enigma ensued. No stack traces would enlighten me, but when I tried deleting the JSESSIONID to trigger auto login, I kept being returned to the login page. After setting many breakpoints, and walking through the spring code, I discovered that there were two keys being set and the curious BadCredentialsException exception being thrown and caught, silently. Shhh. A quick Google search and forum post were not helpful.

This is the trouble with automagical areas of Spring. If they don’t work out of the box, you’d better be shovel ready to analyze some sources. The offending comparison takes place in the following method or org.springframework.security.authentication.RememberMeAuthenticationProvider:

public Authentication authenticate(Authentication authentication) throws AuthenticationException {
	if (!supports(authentication.getClass())) {
		return null;
	}

	if (this.key.hashCode() != ((RememberMeAuthenticationToken) authentication).getKeyHash()) {
		throw new BadCredentialsException(messages.getMessage("RememberMeAuthenticationProvider.incorrectKey",
				"The presented RememberMeAuthenticationToken does not contain the expected key"));
	}

	return authentication;
}

The Fix

The Spring RememberMe docs don’t tell you, but you must set the key in both the TokenBasedRememberMeServices bean and in the tag.

Make sure you define the same key in both places. Update the first tag to be as below to work with my code segment:

<http>
	...
	<remember-me services-ref="rememberMeServices" key="secretKey"/>
</http>

That was easy, but not as easy as a complete, realistic, production RememberMe example on the Spring docs would have been.

Disclaimer: Custom metrics are $0.50 USD a month. Use judiciously.

I want to be able to set alarms when nodes are getting memory starved and to analyze memory trends. Doing this for disk space, heap space, etc is a simple modification of the below tutorial.

Since the AWS docs have no complete example on how to do this, I wanted to provide one. I set this up using nothing but bash and cron (other examples use Python). My goal was a very simple example.

1. Create a new Metric

We’ll skip this since by virtue of putting data into your new metric, CloudWatch will create it for you.

2. Get the free memory

Folks will tell you of vmstat, free, and other utilities, but if you read their man files, you’ll see the value comes from /proc/meminfo, so go to the source. Let’s grep it out of there:

FREE_MEM_KB=$(egrep -o "MemFree:\s*([0-9]*)" /proc/meminfo | egrep -o "[0-9]*")
echo $FREE_MEM_KB

The gist is that we cannot access capture groups directly in egrep, but we can hit it twice with a pipe for the same effect. The -o is used to only return the matching fragment.

3. Pick a Good Name

I found the terms used in the CloudWatch docs a little confusing at first, so here it how to find them. Check on any of your metrics in the AWS CloudWatch Console as shown below:

Namespace

There are existing namespaces like AWS/EC2 and AWS/RDS. Don’t set metrics here, these are for AWS only. Make up a new namespace for your metric like “MySite.”

MetricName

A short descriptive name, preferentially CamelCase. Remember, you cannot delete or change it! Think hard. Note the metric will be expelled after two weeks of disuse.

Dimensions

Let’s say you’ve got a FreeMemory metric in your MySite namespace. What if you want to add this to another instance? You need yet another degree of freedom, Dimensions. In this example the InstandIdis the dimension being used, but you can set others if you like.

Typical parameters found on a CloudWatch alarm

4. Send it to CloudWatch

Make sure you’ve installed the CloudFront Commandline Tools. The README.txt explains it all quite well.

Now just fire away:

FREEMEMKB=$(egrep -o "MemFree:\s*([0-9]*)" /proc/meminfo | egrep -o "[0-9]*")
echo `mon-put-data --namespace="MySite" --metric-name=FreeMemory --dimensions="InstanceId=i-d889e31d" --unit=Kilobytes --value=$FREEMEMKB`

As you can see, we just specify the three degrees of freedom, the units, and the value. Done! And look, we just ran it twice and we’ve already got a chart. Amazon says it can take 15min to see your new metric and 2min to see a new data value. I found these to all happen within a minute, but depends on the ambient CloudWatch load.

There’s the new metric and data points!

Script It

If found the my variables were not always available to my cron user, so I put them in the bash file (below).

#setup variables
export AWS_CLOUDWATCH_HOME=/home/myuser/cloudwatch/CloudWatch-1.0.12.1
export JAVA_HOME=/usr/lib/jvm/jre1.6.0_33
export AWS_CREDENTIAL_FILE=$AWS_CLOUDWATCH_HOME/credential-file-path.template

# get free memory and send to AWS CloudWatch
FREEMEMKB=$(egrep -Eio "MemFree:\s*([0-9]*)" /proc/meminfo | egrep -Eio "[0-9]*")
echo `/home/myuser/cloudwatch/CloudWatch-1.0.12.1/mon-put-data --namespace="MySite" --metric-name=FreeMemory --dimensions="InstanceId=i-d889e31d" --unit=Kilobytes --value=$FREEMEMKB`

And just added the line to the cron file. Note the full paths given since cron does not know these.

# Update AWS custom metric monitors every minute
*/1     *       *       *       *       /home/myuser/scripts/cloudfront_update_metrics.sh

Now enjoy the pretty charts! And set your alarms as needed at a given low memory threshold.

Detailed Memory Usage Chart

Google for invalidating a bunk of CloudFront files and you’ll find many utilities to help you, but did you know you can do it easily through the AWS Console?

Just visit CloudFront > Distribution Settings > Invalidations > Create Invalidation

Now provide the folder and wildcard to blow away a bunch of files!

I.e. images/*

See, and you did not need to write a rate controlled script, pay a 3rd party monthly website, or enter your secret AWS credentials into some shady webapp!

I recently moved my site (www.RunPartner.com) to Amazon Web Services (AWS) from DreamHost because for just a few more peanuts a month I got a ton of enterprise grade services, and the server does not crash randomly any more.

I’m loving AWS, but one thing I wanted to do was consolidate all logs to S3. Let’s say your site gets SlashDotted or Pinned. One of the first failure modes is that your logs swell up, and you’re out of disk space. Since my EC2 instance has just 8GB, this is possible. But why not use that infinite storage pool in the sky, S3? Perfect.

LogRotate

A long time fixture of the log rotation scene is LogRotate, not to be confused with Apache’s RotateLogs. The following script works well for me (two weeks and counting).

The script assumes you’ve installed the amazing command line package, s3cmd.

# rotate the logs!
# common settings
compress
compresscmd /bin/gzip
compressoptions -9
compressext .gz

dateext
dateformat -%Y-%m-%d-%s

rotate 3
nomail
missingok
daily
size 5k
create 640 username username

/var/logs/www.runpartner.com/*.log {
sharedscripts
postrotate
sudo /usr/sbin/apache2ctl graceful

/usr/bin/s3cmd sync /var/logs/www.runpartner.com/*.gz s3://bucket-logs/www.runpartner.com/
endscript
}

How It Works

It took me a few hours to get everything tweaked just right, so I’ll break down the commands for your edification and so that you can customize the script for yourself.

compress
compresscmd /bin/gzip
compressoptions -9
compressext .gz

• compress - enables compression
• compresscmd  – determines the path to the utility used to compress
• compressoptions - command line switches passed to the compression utility
• compressext .gz - this suffix will be used to determine if files have been compressed

dateext
dateformat -%Y-%m-%d-%s

• dateext – enables adding dates to the log file names
• dateformat -%Y-%m-%d-%s - provides the format for the log file name dates

rotate 3
nomail
missingok
daily
size 5k
create 640 username username

• rotate 3 - how many logs to keep locally before deleting. The more, the more space used
• nomail – don’t try to mail the logs to any body
• missingok - tells script not to freak out that there are no files on first run
• daily - rollover logs on a daily basis (still must call from Cron though)
• size 5K – set minimum size of log rollover. If file is smaller than this, it will not be rolled.
• create 640 username username – add any permissions the files should be given on creation. I needed these, or the compression utility did not have the right to compress them.

/var/logs/www.runpartner.com/*.log {
sharedscripts
postrotate
sudo /usr/sbin/apache2ctl graceful

/usr/bin/s3cmd sync /var/logs/www.runpartner.com/*.gz s3://bucket-logs/www.runpartner.com/
endscript
}

• /var/logs/www.runpartner.com/*.log – file selector, can contain wildcards, or can be explicit
• sharedscripts – only run the code between postrotate & endscript once, even if multiple files rotated
• postrotate …endscript - code to send logs to S3
  • sudo /usr/sbin/apache2ctl graceful – gracefully resets the logs on the apache server, otherwise logging stops because LogRotate removed the log file! Some people just kill the pid, but this is much cleaner.
  • /usr/bin/s3cmd sync /var/logs/www.runpartner.com/*.gz s3://bucket-logs/www.runpartner.com/ - use s3cmd to sync your logs (not many of them, so goes fast) to your log bucket on S3.

Cron it

Now just don’t forget the last step!!! You need to add your LogRotate command to cron, so it can run each day. If you have a lot of traffic, you might want to run it on the hour, with your size attribute set so that large logs get moved to S3 quickly, freeing up space on your instance.

# Backup activities
0 0 * * * /usr/sbin/logrotate --state /home/username/scripts/log_rotate.state /home/username/scripts/log_rotate.config

Notice the below I run the rollover at midnight. This way all entries in the log for a given date are really for that date. Also, don’t forget the state file. This is how LogRotate knows what it did last time, so it can decide what to do this time. Finally, notice that cron needs the full path to everything, including the executables and the config/state files.