Tuesday, August 29, 2006

Building the Behaviour Layer

With all of this talk of fancy libraries and so forth, and with the whole Domino web world abuzz with tales of Ajaxian glory, it's sometimes hard to remember that Domino already provides us with a lot of sophisticated functionality. You say you need a field to refresh? There's a checkbox on the Field Properties box to handle that for you. Sure, it takes a post-back to do it, but most web application platforms have done much the same thing since the dawn of time (about eight years or so ago). All Ajax brings to the party is immediacy and fluidity, really. You can make an Ajax app last all day on a single page — something like Writely is meant for that sort of user experience — but the average Domino-suited application (or application suite) is more generally suited to defined "locations". That is, each task type would have its own workspace. Changing pages is not a dread thing to be avoided at all costs.

Okay, what does all of this blathering mean? It means that Domino views, forms and pages created in Designer already do pretty much what our users expect them to do. They just might not do them in the most elegant of ways. For instance, it may take a user several seconds (during which, of course, she has been typing madly away to record one of those elusive thoughts that will never be formulated quite the same way again) to realise that the form has already been submitted for refresh and all of her recent work has gone for naught. That is something that the immediacy and fluidity of Ajax can alleviate.

Now, as proud as I am of the bits of the library I've already put together, I am aware that as it stands, it still requires that a developer's efforts be concentrated on cooking a web application. atDbLookup() is way freakin' kewl (and will be so much kewler when I've gotten the automatic request and callback marshalling bulletproofed), but somebody still has to add it to a form, field, or button event in order to make it work. I really can't wait to see what other people can do with the tool once it has matured beyond the pre-alpha stage. I'm sure there are a couple or three web wonks out there who will build things of exquisite beauty when they don't have to concentrate so much on the mechanics of the app. They will, though, be aware that they are building a web application at every step along the way. Even something like prettyView() takes developer input, if only to add the code to the database and the passthru tag to the $$ViewTemplate. It doesn't have to be that way.

This is where Jack Ratcliffe and I meet face-to-face, so to speak. The Domino Web Tools project was meant to automate a lot of those processes, and I'm pretty sure that having direct Formula analogues handy will make that go a lot more smoothly. As I get deeper into the library project, it's easy to see how something like DWT can become overwhelming and go fallow for a while. Its a Project with a capital P. It was great to see Jack responding to the original library posting.

Right now, when you have "Use JavaScript to generate pages" turned on and select one of the refresh options on a field, Domino generates an event handler like this in the onchange: _doClick("$Refresh",this,null). Overriding a handler like that is very easy to do in JavaScript: document.forms[name].FieldName.onchange = someNewFunction. As long as it's called after the form is loaded, it's nearly foolproof. That means that right now, today, almost anyone reading this has all of the tools they need to create a "behaviour layer" that runs through Domino web page and hangs the stuff they want to hang off of field events, etc. It isn't at all difficult, but it is extra effort, and it is oh, so easy for the behaviour layer (the custom event script for a page) to get out of sync with the form or page design.

That's where the more ambitious part of Domino Web Tools comes in. In the early hours of the project formation, at least, there was much talk of automating as much of the webbification stuff as possible. You know, reading a form's design, say, and auto-generating the replacement parts. Adding datepicker widgets with the appropriate regionalisation to date fields, that sort of thing. It's all of that event code, though, that's the real sonofabee, at least when you have to create equivalent client-side web code. That is, I hope, where this library would make the greatest difference.

There are boatloads of Notes-client-only developers out there, many of whom never, ever stray beyond "pure Notes" applications (if they have to talk to another system, they'll use something like Notrix to do the talking), and who never step foot outside of the comfortable world of Formula Language. Many are only part-time developers — thay have "real" jobs filling up most of their time, and learning the intricacies of HTML, CSS and JavaScript is not something they'll give up their precious family time to do. Now imagine what it would be like for them to be able to create a truly usable, responsive, and (perhaps) even good-looking Domino web application by merely copying a couple of Design Notes into their database (or running an agent against the db to do it for them). Imagine the appearance transformed to look more like the thing they created in Designer. Imagine "Allow values not in list" still allowing a drop-down selection. Date fields with the date selector enabled still having them on the web. View dialogs and name-and-address pickers acting just the way you'd always imagined they should. Post-back behaviours replaced by in-browser updates. Computed fields computing right there in front of the user the way God intended. The one critical customer-interactive application a small Notes shop has does not have to be an aesthetic or functional embarrassment.

Ultimately, that's what I'd like to see happening. As much as I'm jazzed about super-innovative web development on Domino, I think that it may be far more important to the platform to make the low-hanging fruit hang a whole bunch lower.

It's one thing to add calls to a function like atDbLookup to a form. It's a relatively simple thing to make a server request to update something on the web page. But if you use the example I've given before, where the onchange of a field contains something like refreshSelectOptions("FieldName",atDbLookup("cache","","LookupView",this,2)), there is an implied contract of synchronicity. The refreshSelectOptions() function can't do what it needs to do until the atDbLookup() function has returned its values. Well, hell — we've just managed to lock up the user's browser the same way we would have by posting the form for refresh. Maybe not for nearly as long (and thanks to the way I do the caching, not every time, either), but there is that moment where the browser can't do much but wait for a reply. The big "A" at the start of the word "Ajax" stands for "Asynchronous". The object of the game is to notify the user that something is happening, but stay out of their way otherwise. That means divorcing the main activity from the event that caused it, and giving the request all of the info it needs to look after its own affairs when it's complete.

That gets a little dicey when you want to abstract things away from the developer. In normal Ajax development, one would be hands-on enough to know where you are making requests and what sort of response handler you should attach to the request. That is precisely the sort of thing I want to take off of the developer's plate. In the example above, if the lookup isn't in the cache, then atDbLookup() should immediately return something to the refreshSelectOptions() function so it can terminate and let the user get on with life. In the meantime, the lookup can take as long as it takes (that depends on the connection speed and server load), and when it's done, it should be able to complete what refreshSelectOptions() started. Making that work cross-browser and one hundred percent reliably while preserving the entire calling context is one of those easier-said-than-done things. (Return to main text)

Saturday, August 26, 2006

And people wonder why I prefer passthru....

Well, it seems that you folks were able to break my little piece of script pretty darned thoroughly. The "No Documents Found" bit was entirely my fault. Pulling any old bit of script off of the ol' hard drive and posting it with only minimal testing is not a good idea. It worked in the project I was doing, but that's because I had "pinned" documents so there was always something in the view.

The other breakage, though, I would never have picked up. You see, I never, ever, use Domino WYSIWYG to create headings and so forth. There is no semantic value to anything created with FONT tags, so I always make sure that things like that are done in passthru. And I never use the alignment settings, since it creates markup that you can't override with CSS. (The behaviour here was actually better in R5, since CSS could tell the browser to render a CENTER tag with any alignment you saw fit to use. The DIV with an inline style can only be changed using JavaScript, because inline styles will trump whatever is in the stylesheet.)

The problems with the script are twofold. First, if you create a WYSIWYG heading, Domino generates a DIV around the FONT tag soup it needs to render your semantically useless verbiage (sorry, folks, but unless you use an Hx tag to mark a heading, it ain't a heading). That wouldn't be so bad, except that it will eat the passthru DIV you created around the view by putting the heading's closing /DIV tag after your passthru. Easy to fix, you'd think -- just make sure there's at least one line of not-centred text between the heading DIV and the passthru, right?

Well, that does close the alignment DIV tag before the passthru DIV, but it creates a new problem. Domino may throw in an unclosed P tag, just for fun. Now, I have no problem with Domino rendering a line break with a paragraph. It might not be semantically correct when there is no actual paragraph content, but it's not a biggie, either. What burns my butt is the fact that the tag is never closed. That throws the whole document structure for a loop. The P will not be logically closed until its parent, Domino's FORM tag, is also closed.

There are three ways to try to handle this. The first is to futz around with the form in Designer until you can force Domino to render a BR instead of a P. A BR tag is not a container, so it won't interfere with stepping through the hierarchy in either direction. I'm not quite sure what the magic words are, or what face you have to make while doing it, but it CAN be done. Obviously, it's not particularly reliable, and the next developer to touch the form is going to break it. You can also try closing the P tag yourself, but this will get you no further -- eventually, someone is going to do something that changes the P to a BR. Or worse, they're going to try to get rid of the extra space between the heading and the view, closing off the DIV tag yet again.

The second is to expand the script yet again to cover all of the possible bases, searching every nook and cranny of the DOM until the view table, and only the view table, is finally found. You saw how big the original script was. Scanning the whole DOM tree until you find something that looks like it might be a view will make that little script just a little bit bigger. I'm not afraid of a little scripting (remember the library I keep talking about?), but I do resent having to write that much code to get around something that should never have been a problem in the first place.

The third is to lobby IBM for fixes. No, I don't expect them to add an HTML-prettifying function to the view rendering. In fact, it's just about the last thing I'd want to see, since it would mean that "normal" rendering could never be controlled with CSS -- you'd absolutely need to run some script to change the look of what Domino renders. (Yes, I know I'm running script now, but that's to add stuff that doesn't need to be there.) To my mind, something like that would be far worse than the current state of affairs. What I would like to see is a fix that gets Domino out of my way. If I start some passthru on a new line, and that whole line is marked as passthru, I want it to appear in the HTML source exactly where I put it, not inside a tag that Domino needs to render the previous line. In this case, if I wanted the viewPanel DIV to begin inside the heading alignment DIV, I should have to put in in the centre-aligned part of the text. Things like the document selection JavaScript should be moved completely out of the way -- the <head> sounds like a nice place to put that, doesn't it?. And since response-only columns are the last usable column in a response document row in a view, wouldn't it make a lot more sense to colspan the host cell all the way to the end (as this script does)?

No, folks, I am not whining about Domino's overall HTML rendering. While it can be improved somewhat (and Mark Vincenzes and team are working on it for Domino Next), it is very nearly impossible to create semantically-correct markup from purely visual formatting in a reliable way. A human being with good vision looking at the page can intuit the meaning of the text based on its formatting, but a machine can only do so much. What if your headings are smaller than the body text (a common-enough thing in trendy print publications)? There are some things we need to do ourselves if we want the end result to be meaningful and accessible. But there are a few little bugs that need to be squashed. They might not get the fixes into the current code streams (6.5.x & 7.x), but we can try to make sure that they are gone for Domino Next (no fixed number has been announced yet, but it has been stated that the number will be somewhere between 7.9 and 8.1).

In any case, the script as it now stands has been through the wringer. It'll find the view table if it's there and at least comes somewhere after the viewPanel DIV tag, as long as you don't put another table between the view and the div tag. If there's a "No Documents Found", it fails gracefully -- again, as long as you don't put an H2 tag into the middle of the actual viewPanel DIV AND use a centred, justified, or right-aligned WYSIWYG heading right above the passthru. (I'm only looking for the H2, since I can't predict the language of the message.) I've done everything I can to mangle the $$ViewTemplate and the view, and it seems to be bulletproof. Well, at least until one of you folks tries to use it, that is -- somebody's going to have done something I never would have though of trying, and that will probably result in Domino rendering the table cells outside of the table tag or something. Can you ever really win at this game?

Friday, August 25, 2006

Library again....

Working on the Formula Language analogue functions in the JS library has got me wondering. There are several @Functions (well, a lot of them, really) that will return either a single value or a list of values. In Formula Language, that's all fine and dandy, because except in a very few cases, EVERYTHING is a list. That is, you can spend most of your time oblivious to the actual output of the various lines of code you write; you don't need to make special concessions for "text" versus "text list". I've honoured that in the JS functions' inputs -- they can take a single value, an array of values, an HTML field object, or, in combination with atGetField(), a fieldname.

But what about the output? As it stands right now, I am creating the output in an array, then passing the array out if there is more than one value, and passing a scalar if there is only one value. While that's great if you want to cascade these "@Functions", I was thinking that it might make it unnecessarily difficult if you merely want to borrow a function or two to use in your own JS code. For instance, using atReplaceSubstring() is easier than writing something to iterate through two arrays, generate regular expressions on the fly, and do the replacement oneself. But is the return type uncertainty (array or scalar) going to be a problem? Should the return always be an array when the corresponding @Function allows a multi-value return?

For the record, these are the functions I've already completed:

  • function atDbLookup()
  • function atDbColumn()
  • function atExplode()
  • function atGetDocField()
  • function atGetField()
  • function atImplode()
  • function atLeft()
  • function atMember()
  • function atMiddle()
  • function atReplace()
  • function atReplaceSubstring()
  • function atRight()
  • function atSetDocField()
  • function atSetField()
  • function atSort()
  • function atTrim()
  • function atURLDecode()
  • function atURLEncode()
  • function atURLQueryString()
  • function atWord()

As much as I would like to have used a proper "@" in the names, it's an illegal character thanks to Microsoft's conditional evaluation. Okay, the list isn't very long yet, but the atDbLookup() and atDbColumn() functions took a while, as did atGetDocField() and atSetDocField(). All of them rely on XmlHTTPRequests, and the xxxDocField functions need a server-side agent to work. Okay, we're talking about the simplest, smallest web agent ever, but it's still an extra piece. And it will grow as more server-interactive atFunctions are added, like atUserRoles(). Even atURLQueryString() has more to it than you'd think. And don't get me started on atReplaceSubstring(). There are also a few helper functions to do listwise and permuted operations, since JavaScript doesn't know that [text array] + [string] should add the string to every member of the array.

I am also posting this, or something like it, on the LDD forum, but again, your input is valuable to me.

Thursday, August 24, 2006

SntT -- A prettier view

Not everything needs to be Ajaxian. A simple bit of JS can make a standard "Display using HTML" on a $$ViewTemplate look and work a whole bunch better. No hand-coded "Treat as" views, no fancy background requests need apply. Just a passthru <div> tag around the embedded view, this script, and init() in the onload event of the $$ViewTemplate.

By way of explanation for the long listing to follow:

First, I NEVER comment code quite this way in production. The comments here are for the benefit of people who may not be very familiar with JavaScript, and might have trouble following the "story" otherwise. If you want to use the code, do everybody a big favour and delete the comments. If you are at all familiar with JS, you'll probably find it easier to follow the code without the comments anyway. (They do get in the way, don't they?)

Second, there is a lot of explicit use of getElementsByTagName(). I'd never let something like this hit production with all of those wasted characters floating around in there. I use a simple little function instead:

function $tn(tn,el){el=el?el:document;return el.getElementsByTagName(tn);}

So, instead of:

var viewTable = viewPanel.getElementsByTagName("table")[0]

I would write:

var viewTable = $tn("table",viewPanel)

That is a half-truth at best. I'd probably write "var vw=$tn("table",vP)". Or at least have an obfuscator do it for me. I'm not "into" obfuscation, as a rule, but in an interpreted language where the user has to download the source code, sometimes over a bad POTS modem connection, killing off characters is the best way to save the plot.

The following little bit of JavaScript takes an ordinary, Domino-created, "Display using HTML" view and transforms it into something that looks and acts a little more "applicationy". It preserves the selection margin and any clickable headers. Flat? Single category? LOTS of categories? Response docs? Response-to-responses? Action bar? It's all good. Try it out, play with it. You may like it, you may not. And no, Peter, it doesn't mess with the correct functionality when you open and close categories -- the clicked category scrolls to the top if the page is long enough to be scrollable. Sorry about the formatting -- it's gonna be a little bit on the wide side. You'll want to copy and paste this into something that has syntax highlighting (like a JS script library in Designer) -- trying to make it really pretty here makes it too wide for the screen. Even if you have a fifty-incher.

function prettyView(){
  var debugPos = "";

/************************************************
This function adds a whole-row mouseover and click
event to a Domino "Display using HTML" view.
************************************************/

  var panel = document.getElementById('viewPanel');
  //assumes you have wrapped the view in a DIV with an ID of "viewPanel"

  /***************************************************
  Getting to the view may take some work. You KNOW the
  table lies inside your DIV, but Domino may just have
  closed your passthru DIV without asking.
  No, it SHOULDN'T happen. Yes, it DOES.
  ***************************************************/
  
  //Try the easy way first
  var viewTable = panel.getElementsByTagName("table")[0];
  
  //If that didn't work...
  if (!viewTable) {
    //It might be because there were No Documents Found...
    if (panel.getElementsByTagName("h2").length) {
      return;
      }
    //...or maybe Domino ate your DIV for lunch.
    else {    
      panel = panel.parentNode;
      if (panel.tagName) {
        viewTable = panel.getElementsByTagName("table")[0];
        }
      //Of course, the No Documents Found rule could still be in effect...
      if (!viewTable && panel.getElementsByTagName("h2").length) {
        return;
        }
      }
    }

  //First, fix the situation where a collapsed categorized view
  //is all squished over to the left-hand side of the browser
  viewTable.width="100%";
  
  //Then get the collection of rows.
  var rows = viewTable.getElementsByTagName("tr");
  if (rows.length) {
    //We don't want to mess with the header row if it's there.
    //It may contain column resort hinkies.
    //Domino 6 and up renders the header cells as TH elements.
    var startRow = (rows[0].getElementsByTagName("th").length)? 1 : 0;
    for (var i=startRow;i<rows.length;i++){
      //Now, make sure the row is in the view table.
      //Response documents may be rendered in nested tables.
      var grandparent = rows[i].parentNode.parentNode;
      if (grandparent === viewTable) {
        //the first "parent" is an imaginary tbody element
        //so it's row->tbody->table to get to viewTable
        var href = "";
        var titleText = "";
        //Add the mouseover highlight to each row...
          rows[i].onmouseover = new Function("this.bgColor='#ffff99'");
        //...and return to the original color on mouseout.
        //This maintains any alternate colors set in the View Properties.
        rows[i].onmouseout = new Function("this.bgColor='" + rows[i].bgColor + "'");
        //Now get all of the cells in the row...
        var cells = rows[i].getElementsByTagName('td');
        for (var j=0; j<cells.length; j++) {
          try{
            //There is going to be an error at the end of this TRY
            //related to garbage collection of the objects we create here.
            //It is unavoidable.
            //The best we can do is CATCH the error and ignore it.
            
            //We don't want to change the behaviour of cells in the
            //selection margin.
            if (!(cells[j].getElementsByTagName("input").length>0)) {
              //We need to find links to create the whole-row click.
              var links = cells[j].getElementsByTagName("a");
              if (links.length) {
                var count = 0;
                var link = links[0];
                //Not all A tags represent links. We will pass over
                //any named anchors (A tags with a NAME and no HREF).
                while (!link.href || link.href == "") {
                  link = links[++count];
                  }
                href = link.href;
                //We also need to know what's inside the link.
                var children = link.childNodes;
                var testNode = children[0];
                if (testNode && (typeof testNode == "object")){
                  if (testNode.tagName && testNode.tagName.toLowerCase() == "img"){
                    //In this case, it's a picture -- probably a twistie
                    titleText = testNode.alt;
                    }
                  else {
                    //Otherwise, there's got to be text in there somewhere.
                    //It may be nested in FONT tags, and there may be empty
                    //DOM nodes.
                    while (testNode.childNodes.length) {
                      testNode = testNode.childNodes[0];
                      }
                    while (testNode && testNode.nodeType!=3 && testNode.nodeValue!=""){
                      testNode = testNode.nextSibling;
                      }
                    //After all of that, we may not have any text...
                    if (testNode) {
                      //...but if we do, we replace the original link with plain text.
                      var swapNode = link.parentNode;
                      swapNode.replaceChild(testNode,link);
                      
                      /************************************************************
                      NOTE: Those last two lines help the view LOOK a lot prettier,
                      but they also affect accessibility. Keyboard-only users will
                      not be able to tab from link to link. If accessibility is
                      important, comment those two lines out.
                      ************************************************************/
                      
                      titleText = "Click to open " + testNode.nodeValue;

                      /************************************************************
                      For some unknown and unholy reason, Domino renders response
                      documents in nested tables in one column of the main table.
                      Not only does it make this sort of code harder (whine, grumble),
                      but it also means that response docs will shove the main document
                      content over to the right. This will fix that by removing
                      the final cells in the main table's response row and adding
                      their width to the response cell. The rest of the table can then
                      collapse back to normal size.
                      ************************************************************/
                      if (rows[i].getElementsByTagName("table").length) {
                        //This is a response doc, and we are stuck in a nested table.
                        //In order to keep the responses from pushing everything over,
                        //we need to find the outer cell containing the table...
                        var parentCell = cells[j].parentNode;
                        while (!parentCell || parentCell.nodeType != 1 || parentCell.tagName.toLowerCase() != "td") {
                          parentCell = parentCell.parentNode;
                          }
                        //...and work on getting rid of the following cells
                        var killCell = parentCell.nextSibling;
                        var removedCellCount = 0;
                        while (killCell) {
                          //Before removing any cells, we need to find out how wide they were.
                          var oldColspan = killCell.colSpan;
                          rows[i].removeChild(killCell);
                          killCell = parentCell.nextSibling;
                          removedCellCount += oldColspan;
                          }
                        //Now we add the width we removed to the response cell...
                        parentCell.colSpan = parentCell.colSpan + removedCellCount;
                        //...add the onclick event ...
                        parentCell.onclick = new Function("getLink('" + href + "')");
                        //... and the mouseover text.
                        parentCell.title = titleText;
                        //Finally, we change the cursor to tell the user they're
                        //mousing over a link.
                        parentCell.style.cursor = "pointer";
                        }
                      }
                    }
                  }
                }
              if (href != "") {
                //If, after all of that, we have a link location to use,
                //we add an onclick to the cell to take the user to the link...
                cells[j].onclick = new Function("getLink('" + href + "')");
                //... and add the mouseover text.
                cells[j].title = titleText;
                //Finally, we change the cursor to tell the user they're
                //mousing over a link.
                cells[j].style.cursor="pointer";
                }
              }
            }
          catch(e){
            alert(e.message);
            //ignore -- it's because of nested tables on response rows
            }
          }
        }
      }
    }
  }


function getLink(){
  var el=arguments[0];
  if (typeof el == "string") {
    window.location.href = el;
    }
  }

function init(){
prettyView();
}

I try to avoid calling any function in the onload that isn't called "init()" -- that means I can change the function names in JavaScript with a search-and-replace and never have to worry about changing the body onload. The init() function calls the prettyView() function, and the prettyView() function adds an onclick call to the getLink() function. You'll need all three in your JS script library or JS Header.

You can improve the appearance of the view by adding the following CSS:

TABLE {
font-size: 1em;
border-collapse: collapse;
}
#viewPanel TR, #viewPanel TD, #viewPanel TH  {
border-bottom: solid black 1px;
}
#viewPanel TABLE TABLE TR, #viewPanel TABLE TABLE TD, #viewPanel TABLE TABLE TH {
border-bottom: none;
}

You need to set the border-bottom of the TD and TH in order to get any lines at all. The lines won't extend all the way across all of the cells, though, unless you also set the border-bottom for the TR. The border-collapse: collapse; makes sure that both sets of lines look like one. The selectors with TABLE TABLE in them make sure that the response document nested tables don't get multiple bottom borders.

UPDATE:Sometimes I hate this posting of snippets stuff SO much. There was a small problem with what I posted yesterday, in that it came from an earlier version of the original file. (A quick look at getLink() should tell you that it was excerpted from a bigger mess -- it's designed to handle links based on table row ids as well as href values.) The actual, honest-to-goodness production code lives in a template on a server (or group of servers) to which I haven't had access in a couple years, so I had to rely on what I had in text and *.js files here. I gave it a quick test before posting, but then I tested it again, and, well....

The changes live in the little loop where I go looking for the parent cell of response documents. The original code would break if something other than 10pt Default Whatever Plain is selected as the font for the responses-only column. That has been changed so the code will continue upwards to find the containing cell. I've also made the link replacement code two lines instead of one to solve a node resolution problem introduced when looking for the outer cell. For some reason, doing this:

someNode.replaceChild(testNode,link);
was a problem, but doing this instead:
var theSameNode = someNode;
theSameNode.replaceChild(testNode,link);
fixes it. As the code above implies, it's the same node. Not just the same HTML element, but the identical object. The identity check, (theSameNode === someNode), will return true. Yet the two-line version works in every browser I could test, and the one-line version fails in almost all of them. Only Opera, usually the worst browser for complex JS because it swaps engine components when you change its spoofing settings, actually got it right all of the time. Mozilla and IE would bail if the link was on a responses column with a font setting.

I've thrown in the only fix I could think of for the view title alignment problem noted in the comments. Oh, and the <h2>No Documents Found</h2> has been fixed, too (that was in the working original). NOW I know that multiple, seemingly-identical bits of snippetalia on a drive probably means that one is right and most of the rest are just-in-case backups that should have been nuked. Oh, well.

Technorati:

Friday, August 18, 2006

Speaking of huge libraries ...

... is there any interest out there in a general-purpose JS/Ajax library specifically for Domino?

I've seen adaptations of Prototype, Sarissa, Scriptaculous (and even Rico, which is a lovely library but wants everything in its own format) and so on, but would there be a market for a library of components that grok Domino? One that would not force you to write responses that conform to a library designed for other platforms? One that would include not only JS functions that already know about ?ReadViewEntries, categorized views and response documents, but design elements that can make up for the difference between server time and workstation time, that can prevent save conflicts, and so on? One that lets you dynamically refresh fields using, oh, let's say @DbLookup syntax in JavaScript?

If there is any interest, what features would you like to see? Be wild and scary if necessary. There's no law that says I have to include everything everybody could ever want. And I've already got a few tricks up my sleeve that may be as wild and/or scary as what you have in mind. I haven't quite rewritten the Formula Language engine in JavaScript yet, but I have borrowed a few of the more useful list-processing bits, along with their familiar-to-us syntax.

It is my intention to release such a library into the wild. The library itself will be free to use anywhere, both as in speech and as in beer. Thanks to View->Source and browser extensions, there's no good way around that. Even if the code is obfuscated (and it is, has been, and will continue to be, for compactness' sake), a determined developer can copy it and find a way to use it. The API documentation, though, I can charge for, so I will. Or, rather, somebody will.

The documentation will include the clear, unobfuscated code with comments. While the clear code would be entirely impractical as a user download (it is not a little file, or even a bunch of little files), forcing an interested developer to figure out which bits do what in the obfuscated code alone is not going to result in better code or in a better developer. Properly done, this might make a good addition to a certain training materials package, since the doco is as much about "why" as "what".

You see, I'd like to see this, or something not unlike it, become a standard part of the Domino developer's toolkit. While I would like to see every developer equipped with the knowledge and skills to do all of the work themselves, it just doesn't make sense that each of us needs to reinvent the wheel with every new application. Okay, the truth is that I was tired of reinventing wheels by the third go-around. So I've taken the bits and pieces of what I've been doing over the last year or so, located the reusable bits (and refactored to include them where they'd been redone), and come up with something I think is worth sharing. But I know that I've only included features that I needed at the time, and that there have to be things you might think are commonplace that have never occurred to me.

So if you can think of anything you'd like to see, anything that can help you elevate a garden-variety web-enabled database to a holy-crap-gotta-have-it application with minimum effort, let me know. Your suggestion may be what makes the whole thing worthwhile.

How nifty does it need to be?

A while back, Rocky Oliver floated the idea of resurrecting the Nifty Fifty, a group of fifty minimal sample applications that IBM threw into the box with every purchase of Notes 3.somethingorother. They didn't do much, and were never really intended to do much. They were fifty sets of ideas, fifty starting points for developers to build upon.

Lately, the idea has been getting a lot of play, thanks mainly to John Head. John suggested that, with Microsoft entering the fray and including application templates with the various incarnations of Sharepoint, the community should step up and contribute a set of "real" applications to run on Notes and Domino. The community consensus seems to be that the idea would not fly unless the apps were distributed and supported by IBM.

I guess it's time to throw my coupla cents in.

Should there be a set of application templates available that go beyond Discussion, Document Library and TeamRoom? You betcha. Does the set need IBM distribution and support? Again, you betcha. Do these applications need to be everything an organisation could lust after out of the box? Not on your nelly, nor on mine, neither. Not no-how.

What Domino needs is fifty (give or take) Really Good Ideas™ rolled into a neat little package. The applications need to be useful and usable, but each should really only try to do One Thing Right™. They should do that clearly and with voluminous documentation even for the most obvious bits. And while an application would need to include all of the ancillary bits that make its particular Really Good Idea™ work, it should absolutely cry out for extension and customisation.

A big part of what got me started down this track was jonvon's list of suggestions. In particular, it was the suggestion that there should be no obfuscated JavaScript (as one finds in Domino Web Access). As I've been doing a lot of webby things lately, it occurred to me that clear, self-documenting JavaScript on the scale required for a genuine Web Application™, complete with all of the neat Ajaxian features and behaviour layers and everything else that represents large-scale functionality these days, would make the needed libraries huge. Not merely big, but huge. Obfuscation might be a maintenance programmer's worst nightmare, but failing to make the downloadable component as small as it can be by using short names and proxy functions is working at cross purposes to the user community. Sure, it's easier for a developer to see what's going on and make modifications to the code (that is, without having the same obfuscator and replacement key file), but at a cost that can be as much as 80% larger code.

Let's take the Prototype library's $ function as a basic example. The name is impenetrable (and forget about documentation — Prototype was not intended for human developers to interact with, so it's enough that Rails understands what's going on), but by the simple act of replacing document.getElementById("someID") with $("someId"), you save twenty-two characters with every call, recouping the cost of the function in only two calls. Twenty-two characters might not sound like much, but one might make a call like that a hundred times or more in a complex script, and that's more than 2KB saved right there. Short variable names come with similar savings, as do snippet "constants" fed into the constructor of a function or an object.

When it means the difference between 2KB and 10KB, even over a bad POTS connection it's not worth sweating. But when the same ratio takes you from 30KB to 150KB, you start running into usability issues over dialup and on slow, crowded networks. Sure, the file is cached, but that's on a per-database basis (and what if the user is using several applications of the same or similar design?) and only after that first slow download. The fact that the application is better-stronger-faster after it loads means very little if the initial hit feels like one is installing Office. Users will not put up with slow, and it will be Domino's fault no matter who actually wrote the code. Chuck one application, and maybe even the platform it runs on. That means that if clarity and extensibility are among the chief aims of the project, as I believe they should be, then we have to aim lower than Google Maps or BaseCamp with our web examples.

I agree that the templates in the set need to be as simple and clear as they can be. We are, after all, trying to sell the possibilities of the platform, and if a developer can't figure out what's going on, she's going to have a hell of a time trying to build her own custom applications. And they should be no simpler than necessary. Each application needs to do something that will be useful to someone, if for no other reason than to encourage the "Notes guy" to open the thing up in Designer to take a look under the hood. There almost needs to be a hole in every application, though; something that is obviously missing, but not so obvious that the template will be discounted as useless right away. NiftyNext™ should be at least as much about teaching as it is about value added to the platform. Let the all-singing, all-dancing, impenetrably complex applications remain as they are now — commercial ventures and the province of internal corporate developers.

After all, if everything comes in the box, we're all out of work.

Friday, August 11, 2006

From the Mail & Guardian online:

When Australian cricket commentator Dean Jones was fired for calling Hashim Amla a "terrorist", the manne were delighted. After all, if everyone went around indulging provocative and childish stereotypes, the Oom might be tempted to call Jones a livestock-romancing wife-beating string-vest-wearing racist bigot Australian yahoo from the arse end of nowhere whose gigantic mouth is writing cheques his tiny brain can’t cash.

"Dinkum moron" by Krisjan Lemmer, 11 August, 2006

Tuesday, August 01, 2006

@Command([NavigateNext])

First off, I'd like to apologize for still not having gotten back to everyone who has written me. The flood of email has been a bit overwhelming, and I've only recently had the wherewithal to download the messages and compose replies offline (thanks again, Devin).

Among the messages have been proposals for independent contract work, possible positions in and around Toronto, and positions elsewhere. There has been an awful lot to consider, and not just in the philosophical, "you've given me a lot to think about" sense.

A big part of what I had to consider was the simple mechanics of getting back to work. As I mentioned before, my life had taken a bit of a downhill slide, and there is a point below which it becomes exceedingly difficult, if not quite impossible, to recover gracefully. I live in a dark, dank and moldy 90-square-foot basement room in a building (and neighborhood) populated primarily by the drug-addled and the insane. People who are self-medicating in extreme excess, and people who are failing to medicate adequately. The noise, the fights and the screaming can get to be a bit much. Working at home is difficult, but alternating doses of headphones and earplugs make it possible, and the occasional escape to the local intarweb cafeteria is welcome respite. Living a normal scheduled life, though, is pretty much out of the question. I sleep when I can, but I haven't had a stretch of time that would have allowed a full night's sleep in some time, and even then the time wasn't at night. So getting up at a normal time and reporting to work during normal office hours would be hit-and-miss at best. I haven't been particularly successful trying lately.

The fix is a simple one. I just need to move. But moving is expensive and disruptive, no matter how one looks at it. Staying in Toronto puts me in a chicken and egg situation -- I'd need to make a considerable amount of money relatively quickly in order to finance a move that would make a regular job possible, but until I move I won't be able to keep the regular hours that would let me keep a regular job.

My current circumstances, then, are not exactly conducive to a conventional approach. So I've decided on the nuclear option. Killing all of the birds in the vicinity with a single, powerful stone. Relocation. A fresh start in a new environment. New country, new surroundings, new type of work, the whole nine yards.

And so I follow friend Nathan to South Africa. Not to Joburg, though; I'll be heading for Cape Town. And not to the same kind of job, nor to the same sort of pay scale. Heck -- I ain't Nathan, and neither are most of you. But I couldn't ask for a better situation, really -- one foot firmly in the realm of the uber code monkey, and the other in the realm of education. That is, assuming I can manage to complete the seemingly trivial tasks of getting a passport and visa done without learning I'm PNG in a country I'm pretty sure I haven't visiteed before. Oh, and I have to hope that my criminal background check doesn't reveal any new and hitherto unknown details from my blackout days. (The hardest part of a return to consciousness was always hearing about my escapades for the first time. I'm pretty sure I know everything I should know now, but you never know, ya know?)

If all goes according to plan, I'll fill in the rest of the blanks for you soon. I'd still be doing the Notes and Domino thing, but I'd be spending a lot more time doing the things that I do best. And that's already coming too close to saying too much for now.