RSS from Trello with Jenkins

Trello is a pretty nice web site. It is (sort of) a kanban board that is very useful when organizing groups of people in situations where a full agile framework would be too cumbersome. Kanban is used a lot in IT Operations. If you want a great story on it, go check out The Phoenix Project.

One thing Trello is lacking, however, is the ability to tap into an RSS-style feed for one or more of your boards. But, where there is an API, there’s a way. This took me about 30 minutes to iron out, and is heavily borrowed from the basic example in the documentation for trello-rss.

Step One – Three-legged OAuth

Trello uses OAuth. So you will need to get your developer API keys from Trello. You will also need to get permanent (or expiring whenever you want) OAuth tokens from them. This process is a little cloudy, but I found a post on StackOverflow that got me over the hump.

Step Two – a little python

I created a little bit of python to handle this for me. Bear in mind it’s still VERY rough. My though is to start to incorporate other Trello automation and time-savers into it down the road. If that happens I’ll stick it out on github.

#!/usr/bin/env python
from trello_rss.trellorss import TrelloRSS
from optparse import OptionParser
import sys
class TrelloAutomate:
 ''' 
 Used for basic automation tasks with Trello, 
 particularly with CI/CD platforms like Jenkins.
 Author: jduncan
 Licence: GPL2+
 Dependencies (py modules):
 - httplib2
 - oauthlib / oauth2
 '''
 def __init__(self):
  reload(sys)
  sys.setdefaultencoding('utf8')
  self.oauth_token = $my_token
  self.oauth_token_secret = $my_token_secret
  self.oauth_apikey = $my_api_key
  self.oauth_api_private_key = $my_api_private_key
 def _get_rss_data(self):
  try:
   rss = TrelloRSS(self.oauth_apikey,
     self.oauth_api_private_key,
     self.oauth_token,
     channel_title="My RSS Title",
     rss_channel_link="https://trello.com/b/XXX/board_name",
     description="My Description")
   rss.get_all(50)
   data = rss.rss
   return data
  except Exception,e:
   raise e
 def create_rss_file(self, filename):
  data = self._get_rss_data()
  fh = open(filename,'w')
  for line in data:
   fh.write(line)
  fh.close()
 def main():
  parser = OptionParser(usage="%prog ", version="%prog 0.1")
  parser.add_option("-r", "--rss", 
    action="store_true", 
    dest="rss", 
    help="create the rss feed", 
    metavar="RSS")
  parser.add_option("-f", "--file", 
    dest="filename", 
    default="trellorss.xml", 
    help="output filename. 
    default = trello.xml", 
    metavar="FILENAME")
  (options, args) = parser.parse_args()
  trello = TrelloAutomate()
  if options.rss:
   trello.create_rss_file(options.filename)

if __name__ == '__main__':
main()

Step Three – Jenkins Automation

At this point I could stick this little script on a web server and have it generate my feed for me with a cron tab. But that would mean my web server would have to have to build content instead of just serving it. I don’t like that.

Instead I will build my content on a build server (Jenkins) and then move deploy it to my web server so people can access my RSS feed easily.

Put your python on your build server

Get your python script to your build server, and make sure you satisfy all of the needed dependencies. You will know if you haven’t, because your script won’t work. 🙂 For one-off scripts like this I tend to put them in /usr/local/bin/$appname/. But that’s just my take on the FHS.

Create your build job

This is a simple build job, especially since it’s not pulling anything out of source control. You just tell it what command to run, how often to run it, and where to put what is generated.

trello-rss-1
The key at the beginning is to not keep all of these builds. If you run this frequently you could fill up lots of things on your system with old cruft from 1023483248 builds ago. I run mine every 15 minutes (you’ll see later) and keep output from the last 10.
trello-rss-2
Here I tell Jenkins to run this job every 15 minutes. The syntax is sorta’ like a crontab, but not exactly. The help icon is your friend here.
trello-rss-3
I have previously defined where to send my web docs (see my previous post about automating documentation). If you don’t specify a filename, the script above saves the RSS feed as ‘trello.xml’. I just take the default here and send trello.xml to the root directory on my web server.
trello-rss-4
And this is the actual command to run. You can see the -f and -r options I define in the script above. $WORKSPACE is a Jenkins variable that is the filesystem location for the current build workspace. I just output the file there.

Summary

So using a little python and my trusty Jenkins server, I now have an RSS Feed at $mywebserver/trello.xml that is updated every 15 minutes (or however often you want).

Of course this code could get way more involved. The py-trello module that it uses is very robust and easy to use for all of your Trello needs. I highly recommend it.

If I have time to expand on this idea I’ll post a link to the github where I upload it.

-jduncan

 

Starting your project with a super-mirco-mini IT budget. Can it work?

A few months agao I wrote up a series of thoughts (and here, here, and here) about tools that you really shouldn’t wait on when you started up a new project. I totally stand by them.

Today I had a really great conversation with a proven IT/web heavyweight and he’s starting up an IT project with as close to a non-existant IT budget as I’ve ever heard. Their codebase is Python (lots of Django love), so it’s living in Google Project’s realm. Most of their other services (CI is what we primarily talked about) was running on Heroku (http://www.heroku.com/), which at the levels they currently need, is free.

So they’re pretty effectively leveraging IaaS and PaaS offerings at the low, free levels to get their company started. But are they really saving money or are they just deferring the cost with interest like all of our student loans?

With the math I’ve done in the past and recently, IaaS and PaaS are most cost-effective for high-intensity, low-duration operations. At 5AM one of the biggest uses we’ve found for AWS is processing large chunks of scientific data. When you’re dealing with a service you want up 24/7/36[5-6], unless it’s AWFULLY lightweight, it doesn’t end up being a cost-effective candidate for pushing to you “cloud” provider.  I think there are still a few years of truth in these statements. Eventually it will become too cheap to not do it, but that’s a little ways off, still. A company still needs its “40 Acres” somewhere.

So what about these guys? I’m entering into a little bit of speculation here, as I’ve never used Heroku, and I’ve only lightly used Google Project, but both of these are pretty highly customized environments. It would seem to me that the longer you live exclusively in these environments the more your code will become dependent on the idiosyncracies of these environments. I don’t think these environments are bad or undesirable.  I’m just saying that I think they will diverge from a “vanilla” build by their very nature, and bringing an application back from one of these environments to run on bare-metal or a “regular” virtual machine will be increasingly difficult.

So, provided that cost exists, the longer you defer the cost, the higher it will be.  Is it worth it? I don’t know.

If someone has experience either way, I’d love to hear about it. I know I’m going to continue thinking about it…

Python, R and modern bioinformatics

My company, 5AM Solutions, was kind enough to ask me to add a post to the company blog this week, thus no update here as of yet (although there’s about to be). You can check it out over at

http://info.5amsolutions.com/bid/91141/Python-R-and-Other-Open-Source-Goodies-for-Science