This site is a collection of musings, thoughts and ideas that have found their way into my head over the last decade and a half. Hopefully some of it will strike a chord and you can find some use for it. I'm passionate about web application development and I'd like to think that by sharing our thoughts we raise our collective consciousness as a community.
Feel free to check out my musical adventures over at http://music.gabrielbuckley.com/
I am a web application development professional living in Brisbane, Australia. I have over a decade of experience in developing database-driven, web-based applications. My expertise spans consulting, analysis, specification, development, project management, testing, implementation, change management and training.
For more information head on over to the "about me" page.
Often you don’t want to add all the overhead of loading up a big AJAX library like script.aculo.us or YUI just to make a few simple calls back to the server. What you need is a lightweight framework that will able you to make and handle calls back to the server in a simple and consistent manner.
To start with, we’ll take the standard XMLHttp objects common across major browsers and wrap them up in an object:
function AJAXRequest(){
this.req = false;
//Check for a native request object
//(IE7+, Firefox etc)
if (window.XMLHttpRequest) {
try {this.req = new XMLHttpRequest();}
catch(e) {this.req = false;}
}
//Otherwise we might be dealing with an early IE
else if(window.ActiveXObject) {
try {
this.req = new ActiveXObject("Msxml2.XMLHTTP");
}
catch(e) {
try {
this.req = new ActiveXObject("Microsoft.XMLHTTP");
}
catch(e) {this.req = false;}
}
}
}
Now we have our base object, we’ll need to add in a few features to handle the requests. We will want to be able to supply some data to send to the server, we will need to tell the request where to send the data, how to send it (GET or POST) and what to do with the response.
To that end, we create the data, url, method and callBack members for our object and allow the object to receive them as parameters. Our object looks like this:
function AJAXRequest(objData, strUrl, strMethod, funcCallBack){
this.data = objData;
this.url = strUrl;
this.method = (strMethod.toUpperCase() != "POST" ? "GET" : "POST");
this.callBack = funcCallBack;
this.req = false;
//Check for a native request object (IE7+, Firefox etc)
if (window.XMLHttpRequest) {
try {this.req = new XMLHttpRequest();}
catch(e) {this.req = false;}
}
//Otherwise we might be dealing with an early IE
else if(window.ActiveXObject) {
try {this.req = new ActiveXObject("Msxml2.XMLHTTP");}
catch(e) {
try {this.req = new ActiveXObject("Microsoft.XMLHTTP");}
catch(e) {this.req = false;}
}
}
}
AJAXRequest.prototype.objData;
AJAXRequest.prototype.callBack;
AJAXRequest.prototype.url;
And we also want to give it a send() method so that we are able to submit the request to a server, and a method of passing back the response data to our supplied callback function:
AJAXRequest.prototype.send = function(){
var strURL = this.url;
// Check for a valid request member
if(this.req){
// Define an event handler for the ready
// state change event
var _this = this;
this.req.onreadystatechange = function(){
_this.handleStateChange(_this.callBack)
};
// Open the connection using the supplied
// parameters
this.req.open(this.method, this.url);
// Send the submitted data (if any)
this.req.send(this.data);
}
}
AJAXRequest.prototype.handleStateChange = function(objCallBackFunction){
// readyState 4 is "done"
if (this.req.readyState == 4){
// Pass the response from the server back to the
this.callBack.call(this, this.req.responseText);
}
}
And there we have it, a lightweight Asynchronous JavaScript And XML object that you can drop into any page you like. Of course, if your project is using JSON then you can use it for that as well.
Nice freebie from Microsoft, Visual Web Developer 2008 Express Edition puts an (almost) fully featured .NET development environment at the fingertips of anybody who wants it.
So far I haven't found it lacking in any major way as far as ad-hoc .NET development goes, although J# is definitely missed. One feature I particularly like is the lightweight web server that fires up on a random port number to let you test out your ASP pages. No more having to copy test code over to a server every time you want to see how things are progressing.
Overall, an excellent choice for anybody that wants to get a toe in the .NET water, or those who only do a little bit of .NET work here and there, but for hardcore .NET developers it is unlikely to supplant Visual Studio any time soon.
If you haven't had the opportunity to do so yet, pop on over to the eXist website and check out their wares.
eXist is an open-source native XML database. Aside from being free, eXist offers a great little playground for developing XML-based applications. It supports XQuery, XPath, XSLT and SOAP just to name a few.
The main benefit to using eXist is the powerful support for XQuery. Entire applications can be developed just using XQuery Language on the back end. These can then be surfaced using your front-end development environment of choice.
To demonstrate just how easy it is, we'll run through a little app I knocked up in a few minutes to manage a list of albums.
Starting with a sample XML file that looks something like:
<albums>
<album>
<id>000001</id>
<title>The Essential Hank Williams Collection: Turn Back the Years</title>
<artist>Hank Williams</artist>
<label>Mercury</label>
<year>2005</year>
</album>
<album>
<id>000002</id>
<title>The College Dropout</title>
<artist>Kanye West</artist>
<label>Roc-a-Fella</label>
<year>2004</year>
</album>
...
...
</albums>
We can start writing some code to perform specific functions. The functions we will need are:
Starting with the easy stuff, let's look at returning a list of albums:
declare namespace xdb="http://exist-db.org/xquery/xmldb";
let $AlbumList := doc("xmldb:exist:///db/interviewtest/albums.xml")
return $AlbumList
Yep, that's it. Three lines, one of which is just a namespace declaration. Get the XML document we're after, return it to the client. How many times have you spent ages stepping through each record in a dataset, carefully wrapping tags around each field?
Ramping it up a little, let's have a look at the insertion code:
declare namespace request="http://exist-db.org/xquery/request";
declare namespace xdb="http://exist-db.org/xquery/xmldb";
let $title := request:get-parameter("title","Unknown Title")
let $artist := request:get-parameter("artist","Unknown Artist")
let $label := request:get-parameter("label","Unknown Label")
let $year := request:get-parameter("year","0000")
let $AlbumList := doc("xmldb:exist:///db/interviewtest/albums.xml")
let $lastID := xs:integer($AlbumList//album[last()]/id)
let $newID := concat(string-pad("0",6 -string-length(xs:string($lastID + 1)))
,xs:string($lastID + 1))
let $NewAlbum := <album>
<id>{$newID}</id>
<title>{$title}</title>
<artist>{$artist}</artist>
<label>{$label}</label>
<year>{$year}</year>
</album>
let $insert := update insert $NewAlbum into $AlbumList/albums
return <result>
<status>1</status>
<message>Record added successfully</message>
</result>
Again, not a great deal to it. Declare the neccessary namespaces, read in the parameters from the request - giving them some default values if they don't exist, generate a new ID for the record, wrap it all up in an XML node, insert it into the document and return a status message to the client.
The rest of it doesn't get a great deal more complex. The deletion code:
declare namespace request="http://exist-db.org/xquery/request";
declare namespace xdb="http://exist-db.org/xquery/xmldb";
let $id := request:get-parameter("id","")
let $AlbumList := doc("xmldb:exist:///db/interviewtest/albums.xml")
return
if ($id != "") then (
let $Album := $AlbumList//album[id = $id]
let $result := if ($Album) then (
let $update := update delete $Album
return <result>
<status>1</status>
<message>Record deleted successfully</message>
</result>
) else (
<result>
<status>-1</status>
<message>invalid id specified or no album retrieved</message>
</result>
)
return $result
) else (
<result>
<status>-1</status>
<message>No id specified</message>
</result>
)
Simply reads in the supplied album id and, if found in the collection, deletes it, otherwise it returns one of two error messages. Likewise the update code:
declare namespace request="http://exist-db.org/xquery/request";
declare namespace xdb="http://exist-db.org/xquery/xmldb";
let $id := request:get-parameter("id","")
let $title := request:get-parameter("title","")
let $artist := request:get-parameter("artist","")
let $label := request:get-parameter("label","")
let $year := request:get-parameter("year","")
let $AlbumList := doc("xmldb:exist:///db/interviewtest/albums.xml")
return if ($id != "") then (
let $Album := $AlbumList//album[id = $id]
let $result := if ($Album) then (
let $update := if ($title != "") then (
update value $Album/title with $title
) else ( )
let $update := if ($artist != "") then (
update value $Album/artist with $artist
) else ( )
let $update := if ($label != "") then (
update value $Album/label with $label
) else ( )
let $update := if ($year != "") then (
update value $Album/year with $year
) else ( )
return <result>
<status>1</status>
<message>Record updated successfully</message>
</result>
) else (
<result>
<status>-1</status>
<message>invalid id specified or no album retrieved</message>
</result>
)
return $result
) else (
<result>
<status>-1</status>
<message>No id specified</message>
</result>
)
Just walks through the supplied paramaters and inserts their value into the requested album.
OK, so there's the framework. Obviously you'd want to put some validation and security around it before selling to a customer (you would put validation and security on an application before selling it to a customer wouldn't you?) but it's pretty much done. If only everything were that easy.
In this article we will examine how it is possible to give your existing Domino applications some exciting new functionality by enabling them as web services. In particular we will look at:
Domino is, undeniably, an amazing application development platform providing an organisation with a rich depth of features. Providing the data and functionality inherent in Domino applications to other platforms however, can be the source of headaches for administrators and developers. This article will demonstrate a cost-effective way to deliver Domino functionality in the structured and standardised manner encapsulated in web services.
What are web services? Simply put, a web service is a method or function that is available to a client over a network. There are a number of formats that can be used to encapsulate a web service request and response, but the most common is the use of the W3C recommended Simple Object Access Protocol (SOAP).
This picture shows the mechanism by which a client application would call a web service over a network. The client forms the request in terms of a SOAP envelope and submits it over HTTP to the server. The server interprets the SOAP request and performs the functionality requested of it before returning the data in the form of a SOAP response envelope.
The advantages to implementing a web services architecture are myriad:
Looking at the components of a web service, the main transport is the SOAP envelope. All communications between client and server are framed as SOAP envelopes.
<soap:Envelope xmlns:soap="http://schemas.xmlsoap/envelope/">
<soap:Body>
<getProductDetails xmlns="http://warehouse.example.com/ws">
<productID>827635</productID>
</getProductDetails>
</soap:Body>
</soap:Envelope>
<soap:Envelope xmlns:soap="http://schemas.xmlsoap/envelope/">
<soap:Body>
<getProductDetailsResult xmlns="http://warehouse.example.com/ws">
<productName>Toptimate 3-Piece Set</productName>
<productID>827635</productID>
<description>3-Piece luggage set. Black Polyester</description>
<price>96.50</price>
<inStock>true</inStock>
</getProductDetailsResult>
</soap:Body>
</soap:Envelope>
The client application would supply the required parameter(s) – in this case the product ID – and process the information returned by the web service. In order for the client to utilise the web service, it must first be able to ascertain:
To provide this information to the client we must introduce the second component to a web service delivery, the WSDL file. Web Service Description Language is another XML based mark-up language, used to describe web services to a prospective client application.
<definitions>
<message name="getProductDetailsRequest">
<part name="productID" type="xsd:string" />
</message>
<message name="getProductDetailsResponse">
<part name="productName" type="xsd:string" />
<part name="productID" type="xsd:string" />
<part name="description" type="xsd:string" />
<part name="price" type="xsd:decimal" />
<part name="inStock" type="xsd:boolean" />
</message>
<portType name="getProductDetailsPortType">
<operation name="getProductDetails">
<documentation>
Get Product Details Given a Product ID
</documentation>
<input message="tns:getProductDetailsRequest" />
<output message="tns:getProductDetailsResponse" />
</operation>
</portType>
<binding
name="getProductDetailsBinding"
type="tns:getProductDetailsPortType">
<soap:binding
style="rpc"
transport="http://schemas.xmlsoap.org/soap/http" />
<operation name="getProductDetails">
<soap:operation
soapAction="urn:getProductDetails#getProductDetails"
style="rpc" />
<input>
<soap:body
use="encoded"
namespace="urn:getProductDetails"
encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" />
</input>
<output>
<soap:body
use="encoded"
namespace="urn:getProductDetailsResponse"
encodingStyle="http://schemas.xmlsoap.org/soap/encoding/" />
</output>
</operation>
</binding>
<service name="getProductDetails">
<port name="getProductDetailsPort" binding="tns:getProductDetailsBinding">
<soap:address location="/products.nsf/getProductDetails?openAgent" />
</port>
<service>
<definitions>
This shows a simplified WSDL file for the getProductDetails method used earlier. The message tags describe the structure of the input and output SOAP envelopes in terms of their component values and data types. The service tag tells the prospective client application where the web service is located and the binding and portType elements instruct the client to use the SOAP protocol when the service is called and to output a response message when a request message is received as input.
Domino 7 provides some inbuilt features to enable developers to deploy LotusScript functions as web services, but deploying web services in a Domino 6 environment is by no means out of the question. To deploy a web service we need to provide both the service itself and the WSDL file that describes it. This can be accomplished using a Domino web agent as the service and encapsulating the WSDL mark-up as a Domino page.
This shows how a web service would be deployed on a Domino platform using standard Domino design elements. The WSDL Page supplies all the information that the client application requires in order to call the web service. The client formulates a request in the form of a SOAP envelope and posts the data to the Domino server. The Domino server extracts the information from the envelope and passes any parameters to the nominated web agent which then returns the result of it’s processing in the form of a SOAP response to the client application. In order to use the web service, a client application needs only to be aware of the location of the WSDL file.
Ten years of trial and error has brought us one unassailable conclusion regarding the corporate intranet. For an intranet to be successful it must not only be accepted by users, but driven by them.
Just as the World Wide Web is driven by those who frequent it, the corporate intranet relies on the input and management of those who use it on a daily basis. The history of the corporate intranet has shown that it closely follows trends set by the World Wide Web. Early intranets were static, driven by mission statements and followed a top-down, tiered approach to content dissemination. Intranets of today are following the portal-style information aggregate model popularised by sites such as Yahoo! in the early part of the twenty-first century. Organisations can extrapolate this trend to their own advantage today to pre-empt the next trends in corporate intranets.
Looking at the emerging trends on the World Wide Web today there are a number of central themes. Key amongst these is the notion of being driven by the consumer. Blogs, Wikis and folksonomies are replacing the traditional content outlets. The power to contribute content to the internet is no longer solely in the hands of those who can master HTML. Just as current-day internet users host their photo galleries on a flickr.com or photobucket.com account and publish their daily journal on their livejournal.com accounts, next-generation intranet users will demand the same power from the corporate intranet that they come to expect from the web today. Intranets that fail to embrace these technologies – and, by extension, the corporations that manage them – will be perceived as somehow lacking by tomorrow’s technology-savvy intranet user.
The challenges faced by an organisation looking to capitalise on these trends involve as much of a cultural shift in the attitudes of knowledge management professionals as they do an acceptance of technological change. Knowledge management professionals must be prepared to exchange the role of information gatekeeper for one of information facilitator. Organisations need to recognise their employees as valuable sources of information instead of potential sources of disinformation. Editorial control can be relaxed for such user-driven information management to the level that an organisation exercises over water-cooler chat rather than the level required for media releases.
Folksonomies play a very important part in user driven knowledge management. A folksonomy (or folk-taxonomy) is a classification system that reflects the terminology in use by the user-base rather than that which either the business itself or external subject matter experts may impose on a knowledge management tree. The use of a folksonomy in conjunction with – or even in place of – traditional knowledge taxonomies not only allows users to tag content with categories that make sense to them, but to search corporate knowledge bases using contextually appropriate terms and queries. That the folksonomy can be applied across all facets of the corporate intranet makes it a very powerful tool for not only defining terms within the contextual bounds of the organisation but for aggregating content across disparate systems according to the terminology that is most relevant to employees and their job functions.
In practise, facilitating a folksonomy entails allowing users to tag a piece of information with metadata that is relevant to them. Employees of an organisation, as a part of each individual organisation’s culture, share a common vocabulary of vernacular and job-specific terms. A folksonomy plays an important part in firstly capturing, and then leveraging, this information toward a better knowledge management system. Initially this may mean that the same piece of information is tagged with widely differing terms. As more tags are applied to the document – and as more users apply the same tag – the relevance of those tags to the document becomes apparent through attrition and repetition. Unlike a traditional taxonomy, which is devised and imposed, the folksonomy follows an evolutionary path.
Blogs (short for web-logs) are a mechanism by which a user can conveniently keep a journal or log of their activities, thoughts, opinions or anything else they feel like articulating. Blog sites are traditionally distinguished by their simple interface. Publishing a post to a blog is usually a single-click process, allowing the user to articulate, format and publish their entry without the use of complex HTML design tools. The browser experience of a blog site is usually similarly uncomplicated. Navigation is limited to simple categorisation by date and category – allowing the folksonomy again to come into play – with the initial view simply being of the most recent entries.
The ability to allow other users to comment on blog entries is key to providing the user feedback and encouraging communication across individuals with common areas of interest. The fact that peers are providing feedback on their entries encourages blog authors to maintain the rate and quality of their entries.
Forums are another technology available to today’s web user that will be a critical component of the corporate intranet. Forums and associated derivatives such as team spaces provide a collaborative environment for users to discuss issues and ideas while having the outcomes of those discussions remain as a searchable knowledge base for future reference.
Aptana Studio is a great little IDE that I've been playing with quite heavily over the past couple of years. Built on the Eclipse framework it brings together pretty much everything you would want in an Editor.
Code formatting and completion is supplied for HTML, PHP, XML, JavaScript and CSS, as well as the ability to create your own definition files for other languages.
All in all it's a very useful editor, especially having IE and Firefox previews inbuilt.
Agile methodologies have been getting a lot of press lately. Falling somewhere between strict waterfall style development and free-for-all cowboy coding they attempt to bring some method to the madness and let programmers do what they do best.
Although there are differences in terminology, most agile methodologies have a few things in common. Firstly they do things in small increments. This iterative method of software development has distinct advantages. It allows for changes to be made without significant disruption and always keeps a goal in clear sight. That goal is usually to have some sort of working release at the end of each iteration.
The team size for an Agile project is critical. Between five and nine developers is a good size. Any more and it becomes too difficult to keep track of each developer's progress, any fewer and you run the risk of devolving into an unstructured unit. Team selection is also critical. An agile team is no place for inexperienced developers. Everyone is expected to be accountable to the rest of their team and commit to iteration goals. Likewise there is no room for ivory-tower specialists who do one thing and one thing only. Every team member should have at least one specialisation and a good cross-section of general skills that will enable them to be fully utilised each iteration. Keeping the team constant is important. Moving people in and out of the team interrupts the flow of the project and, as such, specialists should only be brought in when absolutely neccessary
Gathering requirements from stakeholders can be a joy or a nightmare. The key determinant of which it ends up being usually comes back to how you as an analyst go about it.
Initially the requirements-gathering phase starts off as a series of fairly convivial meetings between stakeholders who all have grand visions of what they want the application to look like. Unfortunately a fair number of these visions end up being mutually exclusive and the meetings descend into political squabbles that you want absolutely no part of. The key to preventing this lies in a proactive approach right from the get-go. During the first meeting, sit and listen politely, writing down every feature that is mentioned on a post-it note. Once everyone has laid out their vision, put all the post-it notes up on a whiteboard and get your participants to sort, group and prioritise them.
As the sorting process goes on, your application requirements should start to take shape before your very eyes. You will find features that can be combined, as you move through the list, others will become evident that weren't thought of previously as interactions between the different stakeholder's requirements are laid out in a visual format.
Once the sorting is done, you have a visual mind-map of the application structure. It is then a fairly easy process to document, take back to the development team and start putting some time estimates against each component. Once the time estimates have been completed you can take the same document (with price tags attached) back to the stakeholders and give them a chance to re-prioritise their desired functionality.
You know the drill, you have this neat little web app you've been working on and the client says:
Hey, how about you include this weather service I found, you can use a web service right? It's just XML right?
Initially you think yeah, sure, not a problem. I'll just break out my trusty XMLHttpRequest() object and fire off a request, get the XML data back and populate my div tags on the fly. You sit down, write some code and... nothing happens.
A little (or perhaps a lot) of debugging later and you realise that the http request code you're firing off from the browser is returning nothing or, at best, a cryptic error message. And then it hits you. Your site is on this domain, that web service comes from a different domain and the browser's security model thinks you're up to no good.
Short of going back to the client and telling him his dream of being able to see what the weather is like in Murmansk (cold usually) from the comfort of his employee intranet portal is not going to be a reality, there is a solution. Using a little proxy script you can effectively mask the request to the external server so that the browser sees the request as going to the same domain that the web page was sourced from.
Proxy scripts are small, lightweight and barely add to response times (which is fine if your other alternative is a response time of well, never). The code below is for a specific example, but it is possible to write one to take any request and pass it on to the appropriate external service provider. The code below is in PHP but any server-side code will do.
<?php
/**
* @author Gabriel Buckley
* @copyright 2008
*/
$ch = curl_init("http://some.otherdomain.com/getUserDetails.xql");
$strUser = $_POST["username"];
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt ($ch, CURLOPT_POSTFIELDS,"username=".urlencode($strUser));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
curl_close($ch);
echo $output
?>
All we're doing is recieving one URL parameter username and making a request to another server with the supplied parameter. Once that server replies back with its response, we simply echo the response straight back to the client. So nine lines of code to solve one of the most vexing problems web application developers need to contend with. It works by on one hand, using the server as a client to fetch the required data. Then allowing the same process to act in place of the proxied web function call by echoing exactly the same data which satisfies the browser securtiy model's requirement that requests be submitted only back to the originating domain.
I have a over a decade of experience in the IT industry comprising a wide array of roles and with exposure to a number of disparate industries including state and federal government departments. My key focus has been on information and knowledge management, especially with regard to document management, systems integration and business process modelling.
My web application development experience has focussed on creating standards-driven and accessible web applications using core principles of functional and usable design.
I am familiar with a wide range of both development methodologies and platforms and have extensive experience at all stages of the project life cycle. My extensive software development background puts me in the position of being able to adapt to and adopt new technologies quickly and my consulting and analysis experience gives me the ability to rapidly identify innovative methods of utilising technology to enhance business processes.
This site is an experiment in presenting XML content. I have an XML document (which you can view here in its raw form) providing the whole of the content for the site and it is being presented through a lightweight ASP.NET application. Technologies used include XHTML, VB.NET, XML, XPath and CSS.
Feel free to get in touch using any of the following methods: