Summary
Configure applications to send all development emails to developer@localhost via SMTP.

Background

Many applications require the facility to send out emails, and there are many ways of achieving this. However, when a team of developers are involved in creating and maintaining those applications, it is worth defining a strategy that all can use to safely test the email system within the development environment.

Use standard protocols

It is fairly easy to set up an SMTP server on any OS, and so to set up one in any development environment. Therefore, it is easy to set up a development system that will use the same protocols as those used in the production environment. This means that as far as the application is concerned, all that changes between development and production is the address of the SMTP server.

Keep development emails local

Here are three good reasons to keep the emails local:

1. Remove dependence on network infrastructure when testing emails, thereby ensuring that the application’s email mechanisms are being tested rather than the efficiency of the network
2. Reduce the likelihood that test emails will get out of the development environment.
3. Give control to the developer as to how the emails are handled.

.
Use a simple generic mailbox name: developer

If the application framework provides a mechanism to forward all development emails to a single email mailbox, use a generic mailbox name. If you do not use a generic mailbox name, each developer working on the application code will need to change that code to work with their own mailbox. ‘developer’ is an obvious descriptive generic mailbox name.

The developer can choose how to set up their local mail server, and can either create a developer mailbox, or use aliasing to allow another mailbox to retrieve developer emails.

Configure the server to send all emails to a single mailbox

If the application framework does not provide a mechanism to forward all mail to a single mailbox, the local server can be configured to forward all incoming mail to a single mailbox.

Developers control how the emails are handled via the configuration of their local mail server

By using a local SMTP server, each developer can choose to handle the emails in the way that suits them. For example, they can choose to forward the emails to their usual provider, or they can configure an email client to access the local server. The key point being that the configuration they use modifies the local SMTP server, and not the application code.

A Rails example

In this blog post, I describe how I set up my own system to receive developer@localhost emails. Below are the configuration changes, that I used to get a rails app to work with this set up. This code can be used as a template for any rails 3 app:

Added this to /config/environments/development.rb:


config.action_mailer.smtp_settings = {
:address => "localhost",
:port => 25,
:domain => 'localhost',
:authentication => 'none',
penssl_verify_mode => 'none'
}


And added this to the end of that file


class OverrideMailReciptient
def self.delivering_email(mail)
mail.to = "developer@localhost"
end
end
ActionMailer::Base.register_interceptor(OverrideMailReciptient)


The problem
I have never been quite sure what my README.rdoc file will look like until I’ve posted it on github. Using rdoc markup ensures that the README is rendered nicely in github, but getting anchors and indentation to play nicely can be a bit of a fag.

You can generate an html version of the README.rdoc by entering this at the app root:

rdoc README.rdoc

But if you look at all the gubbins that is created along with the html file, it is evident that this is over the top. If nothing else, it creates a load of files you’ll need to git ignore, or delete after you’ve used them.

The solution
Fortunately, the rdocs documentation offered a simpler solution. RDoc::Markup::ToHtml will convert rdoc markup to HTML. That gave me an HTML output, but how do I display it?

Sinatra to the rescue. I’ve posted my code here. I think the code speaks for itself.

To use it, just copy display_readme.rb to your app root, and

ruby display_readme.rb

Then point your browser at http://localhost:4567 and your README.rdoc will be displayed in HTML.

It really is that easy, and that really is a web app in 10 lines of code!

I have started to use Jasmine to provide a JavaScript test environment, and I have to admit that I like it a lot.

I added jasmine to my rails app by adding this to my Gemfile:

gem 'jasmine', :group => [:development, :test]

That adds Pivotal’s ruby plugin to the app. Setting it up is described here.

With that in place, I could start adding tests to my app via /spec/javascripts. Note that the test file name needs to end ‘_spec.js’.

The tests themselves are JavaScript files. The syntax is quite like rspec (which is a not a plus point in my book as I am not a fan of rspec, but I can live with it):


describe("A suite", function() {
it("contains spec with an expectation", function() {
expect(true).toBe(true);
});
});


Note that .toBe is acting on expect, and not the item being tested, so none of the rspec monkey patching issues.

To run the tests, the jasmine gem adds a rake task:

rake jasmine

Running that starts a server on port 8888 (I get a warning ‘WARN TCPServer Error: Address already in use – bind(2)‘ but everything works as it should. The warning appears whichever port I use for the server. I think the server is seeing itself and erroneously issuing the warning.).

Pointing a browser at localhost:8888 then displays the test output.

I found using fixtures was useful. Jasmine fixtures are html pages that you can create with the elements you want your JavaScript to act upon. Which means you can test functionality in isolation and be more sure of what is being tested. To add fixtures, jquery support needs to be added to jasmine. I used a technique described in a RailsCast, to install it. I’d recommend watching the RailsCast even if you don’t want to use fixtures, as it is a nice demonstration of using Jasmine with Rails.

You can see my first Jasmine test here. Now I have a decent test environment, I am a little more confident that the JavaScript in my app is behaving the way I intend it to.

One major web app security issue to be aware of occurs because most object creation and update http calls incorporate the passing to the server of a batch of changes. In rails these are bundled up into the params hash. To make processing the data efficient, the batch of data is handled in one step, for example via update_attributes. The danger is that someone will add unwanted attributes to the batch.

Consider someone sending {admin: true} within the batch of data used to create a new user.

To deal with this, attr_accessible was introduced. This adds a white list of parameters that can be used within batch update methods such as update_attributes.

The problem with attr_accessible is that as it is applied to the model, the white list acts everywhere the model is used; including places where the security is not needed. For example when your want to modify objects at the command console (rails c).

strong_parameters moves the control from the model to the controller. So the white list is applied where it needs to be, leaving behind the scenes operations to use batch commands such as update_attributes, as they are needed.

I’d recommend having a look at the strong_parameters README to see how it works:

https://github.com/rails/strong_parameters

One thing to note is that the two approaches to white listing, do not sit nicely side by side. It is a case of using one approach or the other. In rails 4, you will be able to switch off strong_parameters. This means that you will not have to refactor existing apps to remove attr_accessible when upgrading to rails 4.

However, I think the case for using strong_parameters over attr_accessible is compelling, and therefore would recommend that new apps use this approach.

The simplest way to generate this file is to do so when you create the project gemset:


cd ~/projects/projecta/
rvm --rvmrc --create 1.9.3@projecta


That puts a .rvmrc file in the root of your project. cd out of the project fold and then back in and you will be prompted to trust the new .rvmrc file. Select yes and from this point on, every time you enter the project fold, rvm will automatically load the relevant gem set.

Personally I’d recommend you add .rvmrc to your .gitignore file, as the .rvmrc files includes a build specific ruby version number which can cause problem within groups of developers. Better to manage your own file in my opinion.

ArrayLogic

With my current project, I needed a tool to compare sets of arrays to user defined rules. In other words, a tool that would take a text rule, compare it to an array of objects, and return true where there were matches. The result was array_logic.

My first step was to work out how I wanted my rules to work. For this, I took inspiration from Justin French’s story of how he created formtastic. That is, I sat down and wrote out a set of rules and how I wanted them to work:


RuleSet logic
a1 = Answer.find(1)
a2 = Answer.find(2)
.....
an = Answer.find(n)
rule_set(1) = '(a1 and a2) or (a3 and a4)'
rule_set(2) = 'a1 and not a2'
rule_set(3) = '2 in a1 a2 a3'
rule_set(4) = '(2 in a1 a2 a3) or (1 in a4 a5)'


Once I’d done this, I realised that the functionality could be achieved by working with object ids only. So my objects didn’t need to be ActiveRecord objects or any other complex object. They just needed to have an id method that returned an integer. To manipulate such objects, I could use bare ruby which would allow me to keep my code very simple and self-contained.

I also realised that it was likely that I would want to use this functionality elsewhere. As it would be easy to make a solution as a standalone ruby app, it would be very easy to make it into a gem.

The Thing

One of the first things I did was create the Thing object. This is just a class whose instances have an id method. I added to this a class method make that made it easy to generate sequences of things for testing. I now had an object I could play with to get the functionality I wanted.

Next I needed a set of tests that described how the new tool needed to behave. However, before creating these I needed to create an ruby app framework.

Ruby framework

The framework for a ruby app can be a lot simpler than that required for rails. All I needed was a lib folder to hold my code, a test folder for my tests and a few files in the root. They were:

• licence file: Not required, but a good idea to have one if you want to allow others to use your code.
• README: To start with I put my rules text here. I made it a .rdoc file, as this allowed me to use rdoc markdown formatting to make the text a little prettier.
• A rake file: This allowed me to run a couple of useful rake tasks. Specifically rake test

Namespace

As I wanted to package up my code in a gem that could be used in many different projects, I needed to think about my object naming carefully. I did not want the array_logic classes interfering with objects within the parent applications (or vice versa). The easiest way to avoid this is to use a namespace.

There are two steps to using a namespace in ruby. First to create a module that acts as the name space container. Once this is created, any new class needs to be defined within this module. Secondly ruby expects named spaced classes to be stored within a folder with the underscore version of the namespace name. So I created the module ArrayLogic and put the files containing classes within this name space, in a folder called ‘array_logic’

To put each class definition within the new namespace, I used this pattern:

module ArrayLogic
class ClassName
end
end

I could also have used this style of name spacing:

class ArrayLogic::ClassName
end


I also added to my ArrayLogic declaration file, require_relative declarations to the classes within the name space. This meant that simply requiring ‘array_logic’ from outside the namespace, would automatically provide the required classes within the namespace.

Notice that I also name spaced my test files. So they went into /test/array_logic and were created within the module ArrayLogic. This meant that I could refer to the class Rule rather than having to use ArrayLogic::Rule within each test.

Creation

With that in place, I could start building my application. I started by creating a test for a very simple rule: ‘t1 and t2′, which translated as, “the array must contain an object with an id of 1, and an object with id of 2″. I then built my first version of ArrayLogic::Rule, that would pass this test.

It was then a case of progressively increasing the complexity of test and/or functionality described by the tests and updating my Rule class to suit. In this way I was able to start simple, and gradually develop a solution that did what was needed.

Gem creation

Once I had a version of array_logic that was ready to publish, it was then time to create a gem from it. To do this there were four steps.

1. Set up a RubyGems account: You can set up a RubyGems account here.
2. Create a gemspec file
3. Build the gem
4. Push the gem to RubyGems

Steps 2 to 4 are well described here, so I won’t repeat that. Have a look at array_logic.gemspec to see the options I used.

Notice that I have defined version within the ArrayLogic name space. That makes it easier to manage changing version numbers. You cannot push a new revision of a gem into the same version number. So as your gem develops you need to increment the version number.

Also notice that I have set the home as the github location where the code is stored. That creates a nice link between the gem and the code in the repository.

Using array_logic

The resultant gem is held here.

To use the new gem, I added this to my rails app’s gemfile:
gem 'array_logic' # Used in rule_sets
Note that I added a comment to the gem declaration. I believe this is good practice (see my blog for an explanation).

I was then able to use array_logic in my RuleSet model.

Finally

Creating gems makes it much easier to reuse your code. I find it also helps me focus on how to abstract my code into modular blocks, and as a result I believe I am becoming a better programmer. As you can see above, creating a gem is very straightforward and something I think every ruby developer should be aiming to do.

Formal training has already begun, as ten of our developers recently attended a five day Ruby on Rails course. Although there were no course prerequisites, those who had already started getting their hands dirty with Rails were a lot more comfortable with the course contents than those who hadn’t. Developing in Rails is a bit of a culture shock, especially for those coming from the Microsoft world. Lesson learned for next time – a bit of grounding in key areas before the course would certainly help!

Five days is never going to be enough to cover everything, and so it proved as much of the course focused on Ruby and Object-oriented programming. Knowledge of Ruby (and indeed OOP) is a key requirement for Rails development, but having had only a brief look at what Rails can offer, it is clear there is still a lot to learn.

Discussions that followed the course indicated there are areas we need to work on to ensure our developers can adapt to Rails development. Some were using a Linux environment for the first time and would have benefited from an overview before the course, especially as a lot of Rails work is done using the command line. Due to time constraints we were unable to cover many aspects of Rails in depth – unit testing, data modelling and database management were all mentioned as areas we could do with more understanding in, amongst others. There was a lot of useful information on the course but it sometimes lacked context, and wasn’t always clear when or why we would want to do what was being demonstrated to us.

The aim then is to build upon this knowledge, provide context and assist our developers in building tiered web applications from the ground up.

We will shortly be launching our development community area on Google, which we hope will prove an important resource for our developers and will assist those developing Rails applications, especially those who are new to Rails. We are producing documentation as we go along covering all aspects of application development from the ground up, so developers will have enough information to get started in building their own applications. As our knowledge grows we will continue to update our documentation for the community, while making sure we monitor our code to ensure we are following best practice. It is especially important we document proper usage of ActiveResource (our gem for communicating between multiple Rails applications), as there is a lack of documentation available for writing tiered applications in Rails and this is fundamental to building applications that will adhere to our architecture.

Over time, as the API is developed and our developers become comfortable working in this environment, we hope that future application development will be accelerated as web services will be available and code reuse will be encouraged, with common code available to adapt and reuse as required. We also expect that individuals will naturally gravitate towards specific areas of application development, so multiple developers can work on one application and utilise their skills in the right areas. It might take a bit of time to embrace this new way of working, but we once the initial hard work has been put in we will start to see the benefits of this approach for both our customers and our developers.

“Come on, don’t be shy – one look at these knives and you will see the quality of the workmanship and the sharpness of the edge”
My mum’s pace increased and her frenetic searching brought her towards a large crowd of people in the middle of the store, obviously cleared to allow for more people. Standing in the middle of them, was a short perfectly spherical man, with beads of sweat running down his face.

He put the microphone to his lips,
“This is your one and only opportunity to own this knife, and if you buy today, not only will we throw in a second one completely free, I can also offer, for those smart enough to realise how much these cost, a lifetime guarantee. Yes, you heard me correct – a lifetime guarantee”

The audience whoops and sighs, the ebbs and flows of a pliant swarm of consumers.
Once the price had been announced, a sea of hands holding £10 notes covered the shop floor. People were stretching and pushing to be next in line to receive this wonderful, life-assuring kitchen utensil. I turned to my mother, to find her bagging the newly purchased knives with care.

Now, you may wonder what this story has to do with technology, well it doesn’t so far. It’s what happened a little while later, when we came to try to get a replacement for the knives as they had broken within a couple of months of taking them home.

As we had purchased the knives from within the department store, we felt the responsibility for the purchase fell with the department store. Wrong! It turns out that the knives salesman, had just asked for some floor space, on the promise of getting more people to come into the store. The sales were directly carried out by him, and all offers and guarantees came from him, and not the store. On speaking with the store manager, it appeared that we were not the only ones attempting to claim on the lifetime guarantee. The department store had been inundated with guarantee claims, but the knives salesman was not contactable and had left no forwarding details or onward number. The store manager further commented, “I cannot believe so many people believed the promise of a lifetime guarantee, with such poor quality materials. He’s obviously cleaned out in this town, and is moving onto greener, equally gullible people in another town. It’s ok for him to make the promises and guarantees, when he knows he isn’t going to be around when it’s claimed on and doesn’t have to fulfil the promise. It’s left for everyone else to clean up his mess”.

Is there an ICT relevance in this story? If you rename the buying customers as a Corporation, the department store as the ICT department and the Knives salesman as a New Media technology consultancy brought in to remedy a perceived need, you may start to see a connection. As the New Media Company only has to make the promises to get the contract, delivery is key to their success: long-term sustainability is not. In the case of the Knives salesman, delivery was handing over the knives to his paying customers. Once implemented, the New Media Agency usually demands payment and moves onto pastures new. The Corp. is left with a system that works for today, but only just. As demands, use and technologies change, these delivered systems often reach the tipping point of obsolescence much earlier than other systems, because they were designed with a different set of priorities from those that use the system.

The priorities they were designed with of:
1) Quick production
2) Well presented
3) Easier adoption
4) Simple to use
5) Works for now

were primarily to optimise sales and production. They were not designed for long-term satisfaction or maintainability. Seeing how little care and thought there is for backwards compatibility in the Ruby and Rails world reminds me of the Knives salesman. Whilst we may be lumbered with a system designed at its Zenith, those who created and distributed the code have moved onto pastures new and seem to have little consideration for those using it. Given that most of the code submitted appears to come from one-man-bands and New Media Agencies, having code that works a couple of years from now just doesn’t figure.

Is a 2 year shelf-life for a technology product acceptable? For those on the Apple hamster wheel, undoubtedly. I like to get more value from what I invest my time or money in.

The question, we have to ask as an organisation, is how do we harness the benefits that Ruby has to offer, with an appreciation that there are different priorities from those that contribute code and those that use code.

SOA does not just happen; it is something that virtually everyone contributes to in some way across the whole business. Adopting SOA also calls for commitment, not just lip service but real commitment. SOA does present challenges that can come in many guises and it can be all too easy to slip into practice of stringing a few web services together, building a app(lication), satisfying users and project managers and calling it an SOA implementation. This is all well and good but you are not only just fooling yourself but the business or organisation as a whole. Short term gains like this just plant the seeds of despair. Strong words, maybe but I can understand the reasons as to why this happens, threats to building a SOA come from many directions and it is up to all involved in building the architecture to recognise these threats and take appropriate action. These threats can come from Vendors, the business, governance, staff and that all to important project.

Vendors, in the main are not interested in the architecture – they are more interested in selling you a product. Some only pay lip service to SOA or more simply they just don’t understand it. Simple answer to this, don’t let vendors drive your strategy.

The business can have a poor understanding of what SOA means to them, failure to fully engage the business will just lower the importance of SOA in their eyes. SOA demands support from the stakeholders, they really do need to understand the concept and be fully committed to making it a success, without this any SOA initiatives are doomed to failure. The answer to this means talking to the business, explaining to them what SOA means in a language that they understand. If you do not engage the business then SOA will become marginalised and you’ll find it very hard to get a second chance.

Resistance to or fear of change is a noose that will slowly choke an SOA implementation, SOA does mean change, but IT evolves, it changes  that’s why it’s an interesting field to work in. Adopting an SOA approach will more than likely change the makeup of IT departments. For example people’s roles can change; this should not be viewed as something negative. To me learning new skills is always a plus point. Having the right skill sets helps to avoid the fear of change and it is important that developers have the support in adopting this new way of working. The tools that have been used in the past may not now be right for the job now, Developers need to be made aware of these new tools and working environments. Here in Warwickshire developers are most welcome to visit us and engage in what the authority is doing.

SOA is not a project, a SOA is built by projects contributing towards the architecture. Failure to include contributing to the SOA in IT projects will undermine efforts to build the architecture. Developers and project managers must work together and share the same goal. When deadlines are tight and there is no full commitment to SOA then there is a real risk that the ideals of SOA will be disregarded in order to get the job done.

The lack of the G word (Governance), to most it’s something we treat with contempt, as we don’t like been told what to do, but the alternative is anarchy and that’s what you don’t need when building the architecture. All must stick to the guidelines, deviations from this must properly addressed, not in the form of a witch hunt but more to discover why there was a deviation, what caused it, the impact and if necessary reviewing the rules.

As a footnote to this post I felt that I was writing an article for ‘Relate’ (marriage guidance), but SOA is bit like a relationship where all parties have to be committed to it or else it fails.

Most current IT cost models do not model the true cost of running, developing and supporting an SOA environment, most of these models are been based on outdated predictions and practises. The SOA world is different, it effects how business and organisations function and more importantly it changes how IT functions. The only way a SOA will work positively with in a business or organisation is to have a cost model that accurately reflects the adoption of a SOA environment.

The SOA cost model has two main elements, development costs and the on-going cost of ownership, not to different from the traditional model on the face of it. However, SOA development costs must now understand the relationship between cost and the reuse of services, the benefits of integration in the loosely coupled SOA environment and the benefits of platform independent software design. Not only that the SOA cost model must also embrace the effects of reduced human intervention required in managing applications, as most supporting infrastructure is capable of self-management, we’ve all heard of the elastic cloud. The biggest change to more traditional costing models has to be able to fully understand the true cost of running a service, the bandwidth consumption, CPU time, storage that is consumed etc.

Billing information received from providers should also fed back into the SOA cost model as part of a feedback loop which will help in the fine tuning of the model. However, it must also be realised that when starting out in building a SOA environment costing’s fed back into the model will not be accurate as they will in the main relate to pilot projects, but in time as more services are productionised then accuracy will improve.

As more and more services are delivered into the SOA environment then with the right feedback mechanisms including costs fed back from system support activities i.e. help desks etc., the SOA cost model improves even more and the business or organisation will be able to make better judgements on development costs and the true cost of ownership.

Having a truly accurate SOA cost model will then allow the business or organisation to carryout full meaningful ROI analyses on projects, helping with prioritising developments and maximising on development resource. It will also so help in predicting future demands that will be made of the SOA, thus helping with capacity planning and securing prudent financial deals.

Looking to the future of SOA and adopting PaaS (Platform as a Service) for service provision the hosting of services will get dynamic. Let’s say a business or organisation is consuming platform service from three service providers, we’ll call them Provider A, B and C. Provider B and provider C lowers their costs on bandwidth, then using the SOA cost model the business or organisation could look at what services that were placed with Provider A that consumed large amount of bandwidth and move them to the other providers, thus reducing costs. Thinking more abstractly, what if the cost model understood the sessional effects on service demands are there gains to be had there? Could there be services that do not even need to be running all year round?  Get the model right and thing could get very interesting.