Surya Suravarapu's Blog

Spawning a Process from Hudson

Jul 2

Posted by Surya Suravarapu in Build, Tools | No Comments

It took a little while to figure this out, and hence documenting here ...

Need is to restart JBoss once the artifacts are deployed. I have two different jobs, one for building and deploying artifacts (EAR, in this case) and the other one to restart the server. Former invokes the latter as a part of its post-build actions.

I've setup a build step in Hudson that executes a shell script which essentially invokes stop and start operations on the server. The server stops fine, but when I start the server in the background, server process gets killed once the Hudson process is finished. I've tried multiple ways of achieving this -- tried a couple of JBoss Maven plugins and tried Ant route too suspecting if that was an issue with my Shell script. But the real problem lies with the way Hudson deals with the spawned processes. This behavior is consistent with their design according to Hudson docs.

The suggested workaround for Unix systems is to use something like daemonize. That didn't work for me. Daemonize works fine but the process is still being killed by the Hudson. [Side note: daemonize is a neat tool, glad that I stumbled on it. Need it for some other purposes].

So how was this resolved? Searching the bug tracker, found the exact issue that I mentioned here. The solution/workaround mentioned there works like a charm. Here is what it says --

set the environment variable BUILD_ID to something like 'dontKillMe' in the
process that should stay alive.

Hudson looks for that environment variable when cleaning up stray processes.

SetEnv plugin is already installed on my Hudson server, and setting BUILD_ID variable value worked! May be that Hudson could provide an option on the admin UI for the user to indicate not to kill the intentionally spawned processes.

Share and Enjoy:

Tags: Build, Tools

Terrastore Scala Client

Jun 1

Posted by Surya Suravarapu in NoSQL | 2 Comments

For a while now, I've been an enthusiast following the NoSQL movement and during the process drawn towards Terrastore. Terrastore is based on Terracotta, it's a modern document store which provides advanced scalability and elasticity features without sacrificing consistency. If you haven't heard about Terrastore yet, it's worth checking out.

Sergio Bossa is spearheading the efforts on the project, and he is doing a terrific job at it. I've been in touch with Sergio for a bit and in discussions about how I can contribute to the project. Before jumping into the core aspects of the product I wanted to work on a client implementation so that I can play with Terrastore and understanding some of its intricacies. So I started working on a client implementation in Scala, my language of choice these days.

The client implementation is available at Github. There are few areas that I would like to tighten up a bit in the code, but it's out there for some early feedback.

I'm certainly excited to get started on the Terrastore core, and working with great minds -- Sergio and Greg Luck, a good friend and guru back from Ehcache days.

[Direct link to Terrastore-Scala-Client on Github]

Share and Enjoy:

Tags: NoSQL, Scala

Thoughtworks Technology Radar (April 2010)

May 12

Posted by Surya Suravarapu in General | No Comments

Somehow I missed the release of second edition of Thoughtworks' technology radar [PDF]. As they did in the inaugural edition they evaluated various techniques, tools, platforms and languages and put them in four buckets -- hold, assess, trial and adopt.

I will jump to the biggest surprise (but something that's consistent with what I'm hearing these days) -- GWT is moved back from Assess to Hold status. I used to be a fan of the approach GWT took, mostly coming from my angst towards writing Javascript, and to develop client-side stuff using the language that I'm quite familiar with -- Java. Although I haven't done enterprise stuff with GWT yet, what I heard was it's not as productive as I was earlier thinking about, and this technology radar points to some of the issues with the generation of Javascript in terms of productivity, troubleshoting, not being able to utilize powerful features of Javascript, and unit testing. I'm not giving up yet on GWT, but would certainly like to hear feedback from the folks who used it in production-grade applications with good success.

As far as languages are concerned -- they are excited about two languages that I'm having great fun with -- Scala and Clojure. I'm fairly convinced that the enterprises sooner than later would evaluate options of using these JVM-based languages than sticking with Java alone. Java as a language is not dead yet (in fact far from it and dead is a too harsh a word), but if the current trend continues in terms of these new breed of JVM languages (coupled with some good customer stories) that day may not be that far off. Other than that in the languages section -- HTML 5 is gaining traction, which is expected.

In the tools category -- Restfulie has got a mention. I think this is pretty significant in the world of REST-based development. Restfulie is an excellent tool to achieve Hypermedia constraint (HATEOAS) of REST, which helps significantly in loose coupling between clients and servers. They said it well -- "It [success of Restufulie] is an emperical proof that the web and the hypermedia can be used to orchestrate complex business activities".

Also in the tools category -- Subversion moves back into the Adopt section. It has to be that way all along, in my opinion. I like distributed version control systems (Git and Mercurial) a lot but Subversion still has a place in the enterprise. I second their opinion that it's a solid version control system suitable for most teams. New to enter the radar is Github, another success story, undoubtedly popularized Git along with their source code hosting and social networking abilities.

Share and Enjoy:

Tags: Languages, Tools

Scala: Game of Life

Apr 7

Posted by Surya Suravarapu in Scala | 1 Comment

I'm trying to write as many small programs as possible on my way to learn Scala. Along the lines, Dhananjay Nene suggested that Conway's Game of Life is a good one to implement. So here is an implementation, feel free to critique.

I tried to write it in a more functional way but you would surely see an overlap of imperative programming style (having worked with Java for over a decade now).

Rules of the game

[Reproduced from the same Wikipedia article linked above]

The universe of the Game of Life is an infinite two-dimensional orthogonal grid of square cells, each of which is in one of two possible states, live or dead. Every cell interacts with its eight neighbors, which are the cells that are directly horizontally, vertically, or diagonally adjacent. At each step in time, the following transitions occur:

Any live cell with fewer than two live neighbours dies, as if caused by underpopulation.

Any live cell with more than three live neighbours dies, as if by overcrowding.

Any live cell with two or three live neighbours lives on to the next generation.

Any dead cell with exactly three live neighbours becomes a live cell.

The initial pattern constitutes the seed of the system. The first generation is created by applying the above rules simultaneously to every cell in the seed—births and deaths happen simultaneously, and the discrete moment at which this happens is sometimes called a tick (in other words, each generation is a pure function of the one before). The rules continue to be applied repeatedly to create further generations.

Code

Hopefully self explanatory

Game of Life

object GameOfLife {
  def buildBoard(liveCells:List[Tuple2[Int, Int]], sideWidth:Int):Array[Array[Boolean]] = {
    val board = new Array[Array[Boolean]](sideWidth, sideWidth)
    liveCells foreach (arg => board(arg._1)(arg._2) = true)
    board
  }
 
  def generationalChange(board:Array[Array[Boolean]]):Array[Array[Boolean]] = {
    val newBoard = new Array[Array[Boolean]](board.length, board.length)
    for (row < - 0 until board.length; col <- 0 until board.length) {
      val lives = computeLives(board, row, col)
      newBoard(row)(col) = if (lives == 3 && !board(row)(col)) true
        else if ((lives <= 1 || lives >= 4) && board(row)(col)) false
        else board(row)(col)
    }
    newBoard
  }
 
  def computeLives(board:Array[Array[Boolean]], currentRow:Int, currentColumn:Int):Int = {
    var lives:Int = 0
    for (i < - Math.max(0, currentRow - 1) to Math.min(board.length - 1, currentRow + 1);
         j <- Math.max(0, currentColumn - 1) to Math.min(board.length - 1, currentColumn + 1)) {
      if ((i != currentRow || j != currentColumn) && board(i)(j)) {
          lives += 1
      }
    }
    lives
  }
 
  def printBoard(board:Array[Array[Boolean]]) = {
    println
    board.foreach {i =>
      i.foreach { j =>
        if (j) print("*") else print(".")
      }
      println
    }
  }
}

Test

Let's pass in live cell coordinates and board size to mimic the behavior of Oscillators (Beacon) pattern. See image below:

Test it!

def main(args: Array[String]) = {
    var board = buildBoard(List((1,1), (1,2), (2,1), (3,4), (4,3), (4,4)), 6)
    printBoard(board)
    for (period <- 1 to 2) {
      board = generationalChange(board)
      printBoard(board)
    }
  }

Output

A dot (.) represents an empty cell and a star (*) represents a live cell. Initial, generation 1 and generation 2 in that order below:

......
.**...
.*....
....*.
...**.
......

......
.**...
.**...
...**.
...**.
......

......
.**...
.*....
....*.
...**.
......

[Image courtesy: wikipedia]

Share and Enjoy:

Tags: Scala

Greg Luck’s Ehcache Presentation

Mar 15

Posted by Surya Suravarapu in Caching | No Comments

Few days ago, at Philly JUG, Greg Luck (CTO Ehcache at Terracotta) spoke about Ehcache and Terracotta. The event was attended by over 100 professionals in the area.

History

Greg started off with a brief history on the progress of Ehcache over the years [I still remember those early conversations that I had with Greg about 5 years ago when I first contributed to the project]. The project was never static, in fact far from it, there has been a steady progress from the stage where it was simple-yet-powerful standalone cache to a well-implemented extensible distributed framework.

In the early stages, Greg was not too keen on building a distributed cache. Once the goal of getting out a great standalone cache was achieved, and that coupled with contributions from various people (and feature requests) the quest towards distributed cache began. This post Ehcache goes distributed on his blog explains the thought process of that time quite well.

Founded in 2003 as a fork of one of the other open source caching frameworks (Apache JCS). It progressed steadily with additions like Hibernate integration, web caching, distributed caching, Cache Server with REST and SOAP services. Terracotta acquired Ehcache in 2009.

Ehcache Configuration and Performance

Greg explained the basic configuration of Ehcache using Spring's Pet Clinic as an example -- configuring Hibernate and using Ehcache as the second-level cache. He went over configuration options and about the cache eviction algorithms.

I think the performance discussion evoked quite a bit of interest in the audience (well, who doesn't like talking about benchmarks and pretty charts! ) Arguably, Ehcache is the one of the best out there, in terms of performance. If you want to know more about the tests check out the source code yourself (requires Terracotta user account to browse that repository, registration is free).

Here are a few performance figures from the presentation, showing Ehcache's superiority in terms of performance. The speaker also demonstrated the performance figures of Hibernate read and write with Ehcache as the second-level cache (not shown below). Tests were performed in a cluster of 8 client JVMs (1.75G heap), 1 Terracotta server (6G heap) and using MySql

Get (Read) and Put (Write) performance charts below:

Ehcache in-process vs Memcached:

Ehcache in-process has to be faster than Memcached. If not for anything, the basic setup of in-process should have an upper hand over the serialization and network overhead for the Memcached setup. I'd be more interested in more apples-to-apples comparison of Ehcache server (REST-based) vs Memcached. Mentioned that to Greg and I'm sure he is on to it next ...

Performance conclusions:

After App servers and DBs tuned by the independent 3rd parties -- there is 30-95% reduction in database load, improved 80 times the read-only performance of MySQL, and notably lower latency.

Ehcache Monitor

Greg demonstrated Ehcache Monitor, a console application for management and monitoring of caches. Couple of main goals of this tool are tuning cache usage and detecting errors. It provides information about Cache hit ratios, hit/miss rates, hits on the database, detailed efficiency of cache regions. It also has some administrative capabilities that I still have to explore. The tool is still in beta, but looks promising. Read more about it here.

New Features in Ehcache 2.0

Hibernate 3.3+ caching API: New SPI addresses some of the synchronization issues with the previous versions to better suit for the clustered environment.
JTA: From version 2.0, Ehcache acts as an XAResource and participates in JTA transactions. It detects most common transaction managers for various popular application servers.
Write-through and write-behind caching: Version 2.0 introduces write-through and write-behind caching. In write-through pattern -- cache acts as a facade to an underlying resource, and a write to the cache causes write to the resource behind it. In write-behind the concept is pretty much the same but the writes happen in an asynchronous fashion. Read more here.
Bulk loading: Greg said this is one of the features that's been requested for a while now. Now 2.0 has the ability to significantly speed up bulk loading into caches using the Terracotta server array. Read more here.

Conclusion

All-in-all I think that was a great session, and as always, I enjoy using Ehcache and the association with the project. I also got a chance to hack some code with Greg while he was here in Philadelphia. Hope to contribute more ...

Share and Enjoy:

Tags: Ehcache, JavaUserGroup

Pomodoro Technique

Feb 23

Posted by Surya Suravarapu in General | No Comments

A decade ago Joel Spolsky wrote an article titled Where do these people get their (unoriginal) ideas. What he has written is common sense and most of us who worked for a few years in a corporate environment would agree with (I'm talking based on my IT background). Although the context of his article is different, the underlying theme is higher productivity, precisely the theme of this post. Joel says --

Here's the trouble. We all know that knowledge workers work best by getting into "flow", also known as being "in the zone", where they are fully concentrated on their work and fully tuned out of their environment. They lose track of time and produce great stuff through absolute concentration. This is when they get all of their productive work done. Writers, programmers, scientists, and even basketball players will tell you about being in the zone.

He continues ...

The other trouble is that it's so easy to get knocked out of the zone. Noise, phone calls, going out for lunch, having to drive 5 minutes to Starbucks for coffee, and interruptions by coworkers -- ESPECIALLY interruptions by coworkers -- all knock you out of the zone. ...

As Joel rightly points out in the article getting into that "zone" is not that easy. And once you are in, there are quite a few distractions in a typical corporate environment. Great, if you are lucky and gifted to be get into the zone without much effort. If you are like rest of us, you would be all ears for techniques that improve your productivity.

So an year or so ago, I stumbled on this technique called Pomodoro. At the outset it's a simple tracking and feedback process where the unit of work is the pomodoro, a time slot of 25 minutes. The basic principle is to keep you focused enough for specific period of time (in this case: 25 minutes). Here is a free e-book [PDF] if you are interested to know more.

This technique helped me quite well in accomplishing some of the tasks I have either been not focusing enough or procrastinating for various reasons.

This InfoQ article posed some interesting questions with some counter-arguments from Mario Fusco. Mario compares IT professionals with the professionals from other fields and says --

So, why should our work be so different from the former ones? Why do we always think that our work is so special and unique to need a wide set of specific methodologies? Are we professionals or unexperienced kids playing with something bigger then them? I think that, like any other serious professional, I can stay concentrated on what I am doing for hours. I honestly don't need a pomodoro to keep myself focused for just 25 minutes. And if somebody can stay focused for no more than 25 minutes I am afraid that he should really rethink the way he works.

To respond to that, I'm not so sure whether there is an apples-to-apples comparison between the professions and type of work performed. The basic goal is to be productive. Regardless which profession you are in, you find your own tools and techniques to accomplish that goal. I would treat this technique or something else as a chance to improve. Certainly with corporate emails, phone calls, and other interruptions it's certainly not easy to concentrate for a good chunk of time and being productive (I'm not even mentioning about Blogs/RSS readers, Twitter, etc.). As I already mentioned, if you think you are already productive and can concentrate quite well (without any new techniques), great, pat on your back.

On the drawbacks-front he says --

Said that, in my opinion there are also other important drawbacks in the pomodoro technique. What should I reply to my customer who is calling me, possibly from the other side of the ocean? That I am in the middle of my pomodoro and I can't break it? ...

I don't think the technique is that restrictive. Obviously if that call is important you would take it and discard that Pomodoro, attend the urgent task, and get back to Pomodoro when you are ready. Let's not complicate it more than what it is, this is just a way to being more productive cutting down on the distractions. I don't see this as a hindrance to the team work or on your efficiency to deal with more urgent matters.

Have you used Pomodoro technique, how do you like it?

Share and Enjoy:

Tags: PomodoroTechnique, Productivity

Surya Suravarapu's Blog

Ramblings about software development and stuff related

Spawning a Process from Hudson

Terrastore Scala Client

Thoughtworks Technology Radar (April 2010)

Scala: Game of Life

Rules of the game

Code

Test

Output

Greg Luck’s Ehcache Presentation

History

Ehcache Configuration and Performance

Ehcache Monitor

New Features in Ehcache 2.0

Conclusion

Pomodoro Technique

Twitter Updates

Recent Posts

Tags