Coding my own blog for fun

Hello again! Last week, after moving to new host provider I decided to run my own blog platform, not because I was insatisfied with any of the gazilion blog platforms out there, but my will was to do something more personal and get fun of it.

Said that, and without any subject to talk about, describing the process could be interesting for somebody and as a excuse for a post.

A little context first, I'm a fan of markdown style, I use it for everything, personal notes, technical documentation, and I wanted to use it to write my posts. So basically, I started searching for blog frameworks that support markdown, and found Poet, a micro-blog framework that supports markdown and runs on node.

I've to say that was like a Bingo! moment for me, and it only took 5 to 10 minutes, so I stop searching and start defining what I wanted from my personal blog.

Features

• Clean look
• Markdown format for blog posts
• Searchable content

Poet

Setup Poet is pretty straightforward, the following steps describe the instalation and a basic implementation to get the blog running.

• Install Express and Poet.

npm install -g express-generator
express blog
cd blog
npm install poet --save

• Setup a route to display the posts, notice that posts are being read from the posts folder.

var app = express(),
    Poet = require('poet');

var poet = Poet(app, {
    posts: __dirname + '/_posts/',
    postsPerPage: 5,
    metaFormat: 'json'
});

app.get('/', function(req, res) {
  var postCount = poet.helpers.getPostCount();
  var posts = poet.helpers.getPosts(0, postCount);
  res.render('index', { posts: posts });
});

- each post in posts
  - if (post)
    include includes/post

Basically it reads markdown files from the specified directory and renders to html. All of these steps can be found in Poet documentation.

Unfortunately, Poet doesn't have a search mechanism built-in so I added one. To make that happen, I had to index the content of the posts and then provide a search mechanism that map the search keywords (post content) into the post itself (post link).

To index the content I used Reds, that indexes the content on startup. The following steps describe the installation and implementation of the search mechanism inside Poet module.

• Install Reds.

npm install reds --save

In order to integrate with Poet, a few changes were made to the Poet module to index the content of posts on boot and for the search itself.

• Wrap over Reds API for three simple operations: put, get and remove.

var reds = require('reds')

module.exports = Indexer
function Indexer () {
  this.search = reds.createSearch('blog-index')
}

Indexer.prototype.put = function (key, value)
{
  this.log('indexing', key);
  this.search.index(value, key);
}

Indexer.prototype.get = function (key, cb)
{
  this.search.query(key).end(cb);
}

Indexer.prototype.remove = function (key)
{
  this.search.remove(key);
}

Indexer.prototype.log = function(operation, msg)
{
  console.log('  \033[90m%s \033[36m%s\033[0m', operation, msg);
}

• Instantiate the indexer on creation,

function Poet (app, options) {
  this.app = app;

  this.posts = {};
  this.cache = {};

  this.indexer = new Indexer();

  ...

• Add the code that links all this together,

  ...
    var post = utils.createPost(file, options).then(function (post) {
      return all([template(post.content), template(post.preview)]).then(function (contents) {
        post.content = contents[0];
        post.preview = contents[1] + options.readMoreLink(post);
        poet.posts[post.slug] = post;

        // Add this line to index the content
        poet.indexer.put(post.slug, post.content);
        poet.repository.add(post.slug);
      });
    }, function (reason) {
        console.error('Unable to parse file ' + file + ': ' + reason);
        post = undefined;
    });
  ...

• Then a helper function for search,

 search: function (key, handler) {
      return poet.indexer.get(key, function(err, ids) {
        if (err) throw err;
        var res = ids.map(function(i) {
          var post = poet.posts[i];
          if (post)
            return post
          poet.indexer.remove(i)
        });
        return handler(res);
      })
    }

• Add the search route using Poet API,

poet.addRoute('/search', function (req, res, next) {
    poet.helpers.search(req.query.q, function (posts) {
      res.render('index', { posts: posts });
    });
  })

• The search form,

form(action="/search" method="get")
  input(type="text" placeholder="Search" name="q")

At this stage I was able to publish posts using markdown and search for content, no hassle!

Gimme some Lovin'

I wanted to include some feedback bits like a section for comments, a twitter share and a love counter. Everybody needs some love, right ?!

Comments

To manage comments, and making it happen without too much fuzz, I opted for Disqus, a free service and easy to integrate.

• I had to subscribe the service and then copy the following javascript,

script.
 var disqus_shortname = 'weirdloopblog';
 (function () {
 var s = document.createElement('script'); s.async = true;
 s.type = 'text/javascript';
 s.src = '//' + disqus_shortname + '.disqus.com/count.js';
 (document.getElementsByTagName('HEAD')[0] || document.getElementsByTagName('BODY')[0]).appendChild(s);
  }());

• Add some markup,

  span.comments
    a(href="#") Show comments
    div#disqus_thread

Because loading comments take some time, I delayed the load until someone really wants to see the comments.

• Make comments available on click,

$('.comments a').on('click', function() {
    var disqus_shortname = 'weirdloopblog';
    $.cachedScript( "http://" + disqus_shortname + ".disqus.com/embed.js" );
    $(this).fadeOut();
  });

I must say that now I understand why pretty much everyone is using Disqus, just works!

Twitter share

To keep simple as possible, share a blog post is just a matter of append the unique url to the twitter share endpoint.

• Add the markup link,

li
  a(href="http://twitter.com/share?url=http://weirdloop.org"+post.url)

• Read how many shares this post had,

  (function () {
  ...
    $.ajax({ dataType: 'jsonp', crossDomain: true, url: 'http://urls.api.twitter.com/1/urls/count.json?' + href })
    .success(function(data) { ... });
  }());

Love Counter

To fetch the posts with this information already merge into the post object, I decided to integrate mongoose inside poet, making it easy to decorate the existant post object. The love counter is backed by Mongo database, abstracted by Mongoose module.

• Create a wrap around Mongoose for further use inside Poet,

'use strict'

var mongoose = require( 'mongoose' );

var Schema = mongoose.Schema;
var Love = new Schema({
    title: {type: String, unique: true },
    counter : Number,
    created: Date
});

var Love = mongoose.model( 'Love', Love );
mongoose.connect( 'mongodb://localhost/weirdloop-blog' );

module.exports = Repository;
function Repository() {}

Repository.prototype.find = function (criteria, cb) {
  Love.find(criteria).sort({created: -1}).exec(cb);
}

Repository.prototype.findOne = function (criteria, cb) {
  Love.findOne(criteria).exec(cb);
}

Repository.prototype.add = function (slug, cb) {
  var post = new Love({ 'title' : slug, 'counter' : 0, date: Date.now() });
  post.save();
}

Repository.prototype.increment = function (title) {
  Love.update({'title': title}, { $inc: { counter: 1 }}, function (err, numberAffected, raw) {
    if (err) return handleError(err);
    return;
  })
}

function Poet (app, options) {
  this.app = app;

  this.posts = {};
  this.cache = {};

  this.indexer = new Indexer();

  this.repository = new Repository();
  ...

• Handler for love feedback,

...
increment: function (slug) {
      poet.repository.increment(slug);
},
...

app.get('/b/:post/love', function (req, res) {
  poet.helpers.increment(req.params.post);
  res.send(200);
});

• Get posts to display, and for each one get the love counter (ugly though!)

getPostsWithMetadata: function (from, to, callback) {
      var postsWithMetadata = [],
      posts = this.getPosts(from, to);

      var slugs = posts.map(function (post) {
        return post.slug;
      });

      poet.repository.find({ title: { $in : slugs }}, function (err, docs) {
        if (err) handleError(err);
        docs.forEach(function (doc) {
          postsWithMetadata = posts.map(function (post) {
            if (doc.title === post.slug)
              post.counter = doc.counter;
            return post
          });
        });
        callback(null, postsWithMetadata);
      });
}

Deploy

I made an upstart script that configures the express app as a daemon as well as runs the forever loop to guarantee that every change into the code is absorved without stop the daemon manually.

• Upstart script that configures the PORT and uses forever module,

#!upstart
description "node.js blog server"

start on runlevel [2345]
stop on runlevel [!2345]

script
    export HOME="/home/xxxx"

    echo $$ > /var/run/nodejs-blog.pid
    exec sudo -u root PORT=80 forever /apps/running/weirdloop-blog/app.js >> /var/log/node-blog.sys.log 2>&1
end script

pre-start script
    echo "[`date -u +%Y-%m-%dT%T.%3NZ`] (sys) Starting" >> /var/log/node-blog.sys.log
end script

pre-stop script
    rm /var/run/nodejs-blog.pid
    echo "[`date -u %Y-%m-%dT%T.%3NZ`] (sys) Stopping" >> /var/log/node-blog.sys.log
end script

• Make Poet watch for new entries and listen on specified port,

...
poet
  .watch()
  .init();

  app.listen(process.env.PORT);

TODO

As expected for a few hours project, much still to be done. I'll put the list here to ensure that I'll tackle this issues later, but this list can be endless.

• Provide a clean abstraction to Poet for social APIs.
• Make posts searchable by tags and categories. Poet already support this, I just need to integrate with search.
• Spell check, at least for me non-native english it is helpful.

In the end I get fun of it, so mission accomplished!

Resources

• Poet - Poet is a blog generator in node.js, the base for all this
• Reds - Full text search module for node.js backed by Redis, support my search
• Mongoose - Library that provides MongoDB object mapping similar to ORM for node.js, support my social thing metadata
• Mongo - Document database, support my love counter
• Disqus - Disqus is a free service that enables great online communities, support for comments
• Node - Platform built on Chrome's JavaScript runtime, backed all this