Ted Kulp

Code, Photos, Assorted Nonsense

Send Email to Jekyll

As I posted earlier in my introduction to the new blog, I said that one of the main things I liked about Posterous was being able to create a blog post from an email. It serves two pretty cool purposes for me.

First, it allows me to write a blog post anywhere and send it at will. While I’m not the biggest blogger ever, it was definitely a convenience factor for me. However, with the Dropbox integration, this becomes less of an issue, because I can write a post pretty much anywhere now, including my iPad.

But, more importantly, it gave me a way to send content from other services and import directly into my blog. A blog for me is kind of a collection of rag tag stuff… photos from Flickr and Instagram, articles that I star in Google Reader, etc. Now that I don’t have a common API like I did with Posterous, it becomes a challenge to get content into my blog automatically. I’ll go more into how I got around that later on.

Getting Email In

Since I have a dedicated Ubuntu server for several sites of mine on Linode, I have the power of being able to hack up Postfix to bend to my wills. This probalby isn’t the case with everyone, but if it does apply, please read along.

The idea was to create a hidden email address on my main domain, take emails from that and get them to a ruby script.

The first thing I did was create a quick and dirty script that takes an incoming email over stdin and processes it into the correct directory (either _posts or _drafts, depending on tags – more a in sec). The way I tested this was just to take a full email that I sent to Posterous earlier, grab the source of it (Show Original in Gmail), and put that into a text file. Then just pipe that into my script (cat email.txt | receive-email.rb) and I was good to go. Sure I could’ve created a test suite, but like I said, it was quick and dirty.

Some things of note, here. I’ve taken the Posterous system of being able to put metadata tags right into the subject line. So, it uses the majority of the subject as the post’s title, and filters out any tags that look like: ((key: value)). Right now, it handles tags ((tags: tag1, tag2)), the ability to use drafts ((draft: true)), type of markup: ((markup: html)), slug ((slug: this-is-the-slug)) and layout ((layout: post)).

Files are written out to the appropriate dropbox folder, and picked up automatically to rebuild the site. I’ll describe that in detail in another post.

Postfix config

It took a bit of playing to get this working in Postfix, but I did eventually get it. Without going into crazy detail on stuff that’s already been rehashed several times on other posts, let’s just say that putting a script into your alias file is a bad idea because the script will be run as root. It’s better to use the transport method to pipe emails to a particular script – sort of how something like Spamassassin would be setup.

In your master.cf, you would add another handler to the bottom similar to this:

1
jekyll unix - n n - - pipe user=tedkulp argv=/path/to/jekyll/email-script/receive-email.sh

It very simply takes any emails with the “jekyll” transport and pushes them to the script that I setup. Notice it’s an .sh and not the .rb. This was because I have rvm setup globally, and it’s a bit difficult to get the environment params correct. Basically, it’s just setting the environment and pushing all of stdin to my script – no big deal.

Because I’m hosting my tedkulp.com domain on this server as well, I can’t just push all of my email to jekyll. So, for my hidden email address, I use a trick in my virtual file to rewrite it to an unusued subdomain, which in my case is hidden.tedkulp.com. My line in the virtual file looks something like:

1
hidden-email@tedkulp.com hidden-email@hidden.tedkulp.com

This is only rewritten for the purposes of Postfix, and doesn’t translate to anything else. It’s only purpose is to allow me to write a custom transport rule for hidden.tedkulp.com.

The last trick now is to setup my transport file to take all emails to hidden.tedkulp.com and push them to jekyll. Since it’s not a real subdomain, it’s only going to see emails from my rewritten hidden email address. It looks something like:

1
hidden.tedkulp.com jekyll

A quick postfix reload, and you’re good to go.

If then what now?

Now for the issue of the automated posting. Since we have our email address to send content to, we just need something to do it for us.

There’s a new service that’s still in early beta that is making that process a WHOLE lot easier. IfThisThenThat (lovingly called ifttt) is a service that allows you to take triggers from one thing and apply them to another thing. So, basically, you can make a task that will, say, email your hidden email blog address everytime you create a new photo in Instagram. It’s stupidly powerful, totally nerdy, and downright awesome. They’re really responsive on twitter and very receptive to new ideas, which is great. I’ve been using it for a few months now, but it really didn’t becomes THAT useful until the blog switch.

ifttt Tasks

Here is an example of the template for the email that’s sent from my Instagram task. Simple as can be…

Instagram Task

Now, I have all of my automated posts emailed to me to my secret email address. Awesome!

In my next post, I’ll discuss how to get my posts from Dropbox to automatically show up in Jekyll without touching the sever.

The New Blog

A little while ago, while listening to Build & Analyze, Marco and Dan we’re talking about the idea of having a static page blogging system. It was a fairly simple concept… Store all your posts in Dropbox, and have a system that could automatically generate a full website based on those posts.

In a way, it makes a lot of sense. Blogs started out as static sites, and only moved to dynamic code when people wanted things like comments or trackbacks on their site. But in the days of Disqus, it doesn’t really make a ton of sense to have a whole site regenerate a site from static content. Add the fact that I get very few comments, and no trackbacks, it’s really just a waste of processing power.

My site has been hosted on Posterous for awhile now. And I’ll be honest, I really have nothing against them. If someone was going to go into blogging for the first time, they’d be near the top of my list – it has a lot of great features. But, it also has a few small nagging items – which wouldn’t bother normal people. But, I’m an nerd, and nerds aren’t normal.

Marco was starting from scratch, which is fine for him. Since I knew that I was going to probably add a lot of stuff to this system (and since Marco still hasn’t released his code for his yet), I decided to do the smart thing and look for some other system to build on top of.

Since I’m living the Ruby lifestyle these days, I decided to start messing with Jekyll. Jekyll is great. It does the bare minimum of what I needed to get started. It takes a directory of posts and creates a static site around them. It handles Markdown, it has a basic plugin system and it uses a decent template language. So after exporting all of my posts from Posterous, I proceeded to start hacking the thing up for my benefit.

There were a few things that I really appreciated about Posterous that I wanted to get into my own system. The first was the ability to create blog posts via email. While I didn’t always use this, I really liked the idea and did post about half of my articles that way. It also is a very easy way to automate posting from other systems (I’ll get more into that later). The other thing was the Autopost system. I loved being able to create a post, and have it show up in Twitter, Facebook, etc. Of course, they have a ton of infrastructure dedicated to this system, but I at least wanted to hack together a very basic system.

After roughly two weeks of hacking here and there, I’ve changed my site to point to my Jekyll blog instead of Posterous. While the site isn’t completely done (will it ever be?), it’s at least far enough along that I felt it was ready to go live.

Over the next couple of weeks, I’ll do some posts that give specifics on how it all goes together. It’s a fairly complex system, and while totally unnecessary, it helped me learn a lot… and that’s what it’s all about.

If you’re interested in the code that’s gone into this, you can check out my Jekyll Fork, my templates and plugins, and my email-to-jekyll script.

BREAKING: Sony PlayStation Network Coming Back Online

Sony has announced that it has just restored some PlayStation Network services after hackers forced Sony to take it down.

The announcement was made by Sony Corporation Executive Deputy President Kazuo Hirai. “The services are being restored in phases, and I am pleased to say that the first phase has been launched in most regions around the world,” Hirai said in a video statement.

Sony rebuilt the entire PlayStation Network following a breach of its security that resulted in the compromise of 24.6 million user accounts. The changes are being delivered in a new system update that is restoring online gaming and Qriocity services.

“We have greatly updated our data security systems,” Hirai noted in the video statement. “These changes were the result of an intensive investigation aided by some of the most respected forensic and security experts in the computer industry.” Those changes include more advanced security technology, advanced levels of encryption, additional firewalls and better early-warning systems, he said.

In addition to enhancing security, Sony is also giving PlayStation Network and Qriocity customers a free month of service. And while these measures are a good start, Sony has a lot of work to do to regain user trust. “Please know that we’re doing everything we can to fully restore network service around the world and to regain your trust in the days, weeks and months to come,” Hirai stated.

Reposted from Mashable!

New iPhone to Be Called iPhone 4S? [REPORT]

Speculation continues to swirl around the next iPhone, and now an analyst claims he has new information about the highly anticipated smartphone from Apple.

It will be called “iPhone 4S” when it’s released in September, according to Jefferies & Co. analyst Peter Misek. He adds that Apple will announce “distribution deals for the iPhone with Sprint and T-Mobile in time for the holidays.”

In addition, the new iPhone will have a dual-core A5 processor under the hood, helping it keep up with numerous dual-core smartphone competitors already on the market such as the Motorola Atrix 4G and HTC Sensation.

http://mashable.com/2011/05/14/iphone-5-report/#

AJAX W/ Silk Framework

Silk is finally getting to that position where it’s about ready. All of the pieces that I’ve been putting together over the past few months are all starting to fall correctly into place and it’s getting usable.

I spent a lot of time over the last week working on the display portion of the system, mainly views, controller output, partials and javascript. Still a bunch of work to do, but I at least have results to work — now I can just refine.

Last night, I tested the concept of doing a very simple AJAX form, and it went better than I had hoped. Here’s the recap…

Create a controller. Being unoriginal, I called it TestController.php. It would go into the app/controllers directory. It would look something like:

use \silk\action\Controller;
use \silk\form\Form;

class TestController extends Controller
{
  function index()
  {
    $form = new Form('test_form', array('remote' => true));
    $fs = $form->addFieldSet('field_set', array('legend' => 'Config Options'));
    $fs->addField('TextBox', 'field1', array());
    $form->addButton('Submit', array('disable-with' => 'Submitting...'));

    if ($form->isPosted())
    {
      $form->fillFields();
      if (!endsWith($fs['field1']->value, 'was here'))
        $fs['field1']->value .= ' was here';
    }

    $this->set('form', $form);
  }
}

The 10,000 meter view is that we’re basically creating a form object. What makes it an ajax form is simply the 'remote' => true in the Form constructor.

Silk is using a style of javascript called UJS, which was made popular by Rails. Without going crazy w/ the details, the idea is that various hooks to the javascript can be created by simple HTML5 data-* tags embedded into the html. The remote basically adds a data-remote='true' to the form tag when output, and the UJS javascript takes it from there.

After that, we check to the see if the Form was submitted. If it was, we fill in the Form object with the values that were posted. Then, it’s just a simple check and append of “ was here” to the value that was input. Last, but not least, we set our form object to be visible to the views/template engine.

Next, create yourself a default view in app/views/test/index.html.tpl. The naming is important here. It’s basically saying that you want to have a view that a. responds to the index action, b. responds to only html requests, and c. is a Smarty template. It can very simply look like this:

{partial name="_form"}

This just renders a partial template called form in a div. This partial would be in app/views/test/_partial.tpl (it’s a Smarty template, and responds to all actions — html, js, etc) and would look something like:

{$form->render()}

Yup, that’s it. This takes the $form object set by the controller, and renders it to the browser with all of the form rendering defaults. Not fancy, but functional. Now, if you go to http://my.silk.vhost/test, you should get the form in all it’s glory.

However, if you fill in the text and hit submit, it won’t do anything. At least it won’t look like it, because the framework will call a 404. The reason is because the main view (index.html.tpl) is only looking for html requests, and since we set remote to true in our Form, it’s sending a AJAX=y ‘js’ request instead. Since the view doesn’t response to js requests that are beneath it, it says nothing is there and moves on.

In order to return back our updated form declaring that “value was here”, we need another view specifically for the js request. We would call that index.js.tpl, and it would look like the following:

$('#form_me').html({partial name="_form" js=true});

Again, yes, that’s it. We call our same partial, appending the js=true parameter. Behind the scenes, this runs the output through json_encode so that we don’t have to worry about improperly escaped values being sent back to the browser. The beauty here is that we’re using the same exact partial in the same exact way as the html version. The only difference is that we’re dynamically replacing the contents of the #form_me div w/ jQuery instead of refreshing the page.

Untitled


Taken at Christian Life Center