30 December 2008

Teaching Servlets and JSP

It's the end of a week teaching Sun's SL-314 course -- "Web Component Development with Servlets and JSP" in Cape Town. It's been challenging and demanding, but, above all, fun! I had a great bunch of participants on the course, a couple of whom were old friends from an "Intro to Java" course (Sun's SL-275 course) some months ago.

The course materials are pretty good, although with a couple of misfeatures -- in my ever-so-humble opinion, as always -- and one glaring omission! (Never mind the details; I won't bite the hand that feeds me. This time. Rest assured that I've passed my thoughts on to the good people at Sun Education.) However, the whole sequencing and structure of the course has set me thinking, "How would I go about structuring such a course? What examples and challenges would I use for lab work?"

I take this teaching stuff pretty seriously1. So I'm always looking for ways to add value above and beyond what is offered by the course materials. Of course, the most valuable thing a trainer can bring into these sorts of skills-oriented technical courses is one's own experience actually using the technology on real-world projects. But beyond that, a lot of value can be added (or subtracted) by the sequence, timing and manner in which the material is presented. In a 5-day course time is terribly limited, so practical examples and labwork are necessarily very constrained, so it is very hard to work-up really good examples.

Setting examples and lab-work for courses is actually a very hard problem. You don't want to restrict the work to toy problems -- students pick up on that very quickly, and, even more quickly, lose interest in the course material, and, more importantly and with longer-term consequences for the remainder of the course, they lose respect for the course (and sometimes for the instructor, too.) On the other hand, time is very limited -- perhaps a couple of hours for any given issue, so extensive examples take too long to implement.

Then, too, one is in the position of trying to illustrate and/or reinforce key points in the course material, and developing actual code always involves a measure of extraneous detail -- housekeeping code. It adds nothing to the teaching purpose of the exercise, but it certainly chews into the time available!

And -- not to be neglected -- I think it is important that examples be at least a little bit entertaining. After all, we can do order-entry at work anytime... But there's a borderline across which "entertaining" becomes "irrelevant". (Much the same can be said of micro-benchmarks.)

Relevance? You wanted relevance from a blog post? As opposed to rambling?

OK.

I'm starting to put together my own material for a number of courses, centring around the gaps in the corporate professional training space (OO and Java technical matters, of course.) that I've been seeing over the past decade.

(Of course, now is a brilliant time to be putting my own course-material together. In times of economic downturn, the very first thing to get the axe is training! How short-sighted is that?)


[1] As with most things, the more you put into it, the more you get out of it!

15 November 2008

Brain Delayed - In Transit

On my way down to Cape Town -- yet again -- for a Java Intro course -- yet again...

On arrival at the checking-desk, get told that I'm not booked to go to Cape Town, but am eventually found to be on my way to George from Cape Town. Returning to Cape Town on Friday. Wow! What happened here? OK, so off to the booking office where I have to fork out an extra R700 to cover the ticket change. Fortunately the SAA staff at George and I know each other by first name. (As a result I seldom have to produce ID to check in -- most of the staff can ID me by sight. Beat that for personal service! ;-)

Sudden thought: Let's check the car rental reservation. Sure enough! I'm booked to pick up a car in George. Get that one changed. Things are so screwed up at this point that I called the B&B where I'm allegedly booked to stay in CT. Yes, indeed, they DO have a reservation for me.

WTF was somebody at the travel agency thinking? This guy is going to fly from Cape Town to George, pick up a car there, and then stay 5 nights in Cape Town before reversing the journey?

Now I only have to waste an extra hour-and-a-half in the airport. I just love hanging about in airports, don't you?

25 September 2008

Back in the Saddle: Java 101

Interesting to be back in the Training Game. It is a Java 101 (Introduction to Java Programming for Programmers) course. With some differences.

Its all for one corporate client. My God, its been a long time since I had to deal with Corporate IT Thinking. There's some of it that exists for good reason. And then there's a whole lot of crap. Stuff that's more to do with politics, staking out of turf, ass-covering... The real downside for a Java 101 course is not so huge, but still... everybody is from the same organisation, though they might well be from different planets, considering the departments they're drawn from... But, there's still not a lot of cross-pollination of ideas. The whiff of groupthink.

I've taken the format of the "usual" 5-day course and adapted it somewhat to a 6-day format split over two 3-day sessions a couple of weeks apart, so there's a lot more breathing room, but, for any "Intro to X" course there is a certain body of knowledge that has to be covered, and there's a certain Logic to the order in which material has to be presented, so there's no escaping... There's just a bunch of stuff that has to get taught before people can realistically handle a tutorial (a.k.a. "homework".) All resulting in a pretty heavy First Three Days. I've just finished Day 3 ("Subscriber Trunk Dialing".) I'm exhausted.

Differences: In the usual "5-day" format I'd have to be really winding up my energy to stay the pace for Day 4. This time around I get a week-long break, and get to start fresh. Mmmmm... Gooooood!

Happily I have a fairly bright bunch. Quite a mix of backgrounds, all the way from a university graduates to "just-a-programmer-can-I-google-a-solution-and-cut-n-paste-the-code". The usual chirpy heckler who borders, at times, on irritating, but helps keep things alive. The usual "deeply tech" who catches all my mistakes and omissions, and keeps me on my toes.

I'll confess that I'm having a ball!

11 September 2008

Infrastructure Fix

Finally!  Took a day off from preparing for the (Advanced Architecture and Design) course I'm hosting next week, and spent the time getting my local infrastructure back on its feet after a breakdown some weeks ago.  Basically the LAN server died... motherboard or CPU. Suddenly we realised just how much we've come to depend on that little machine!  DNS caching, plus names for all the machine on our farm-LAN1, HTTP proxy, local Subversion server, file-share, music streaming, and not to mention supplying the base-infrastructure for some Jini services that come and go as we play around with Jini and JavaSpaces.

So now that we've beefed-up the CPU and disk a bit, it was "just" a case of getting all the bits of server software installed and cleanly configured. Nothing difficult, but it all takes time! Especially when you're a bit.... fastidious?... about getting the config "right", as opposed to "just poking things until they work". I truly despise "voodoo config" where people don't really make the effort to understand the impact and effects of what they're doing, and fuck it up royally as a result.  "Oh, well, I'll install mod_disk_cache and mod_mem_cache... after all, two caches must be better than one, mustn't they?"  Ding!  You Do Not Win The Lounge Suite!

Well, its been fun! And finally I can forget all about it again until the next hardware failure. The only thing I really would like to achieve is to get the server quiet2, and get its power-consumption way, way down!  And maybe have a few more tiny-little servers...

[1] The term "server farm" takes on a whole new significance ;-)


[2] Yeah, I'm a complete arsehole about noise. I really, really, really hate noise...

22 August 2008

Courseware: The Next Step

Just received my Instructor's Manual (only a week late!) for Sun's SL-425 "Architecting and Designing J2EE Applications"1 and I'm very happy to see that it is basically the same course as the old "Architecture and Design" course I taught several times lo' those many years ago when I was so frequently on my feet as "Herr Instructor".

This is easily the best2 course I ever "taught"! It is aimed a senior, experienced designers and developers3, and confronts head-on the sticky few-good-answers stuff, the ill-defined and the fuzzy grey areas. I have always run the course in a round-table "workshop" format. When you get 8 or 10 senior developers into a room, each with a decade or two of experience, you're not going to be their Teacher. And you sure as hell better not have a tender ego. "I don't know" is a frequent answer. The job is much more one of facilitation: Keeping discussion on-track, drawing quieter participants into the discussion, acknowledging expertise and encouraging people to share their (often vast!) experience. I found it hugely enjoyable to engage with seriously expert people, and to facilitate drawing out their expertise. And I learned a lot!

It has been a very long time that I have wanted to run this course again, so I'm looking forward to it hugely!

So: I have a couple of weeks to catch-up on the changes since last it taught facilitated this course. Mainly the technology has caught up with the tech: Where there was only CORBA or RMI a decade ago, there are now a host of J2EE technologies, and the course now includes them. Where a decade ago the whole idea of enterprise-architecture patterns was pretty new and unheard-of, now there's the J2EE Blueprints Catalogue (even if those yanks can't spell "Catalogue"!)

Of course I'll probably be unable to resist the temptation fo slipping in a few sly teasers about Jini and Javaspaces -- but Hey! Sun seem to encourage instructors to go beyond the boundaries of the course material, and encourage us to bring our own experience into the classroom. Or workshop in this case.

I am still thinking about floating my own version of this course, perhaps a little less attached to Java tech, so that more people might engage. The absolutely best runs of the course were when we had many people from different organisations and backgrounds. Cross-pollination really works. Ask plants!
----
[1] Is there such a word as "Architecting"? My spelling-chequer doesn't seam to think sew.


[2] "Best" from my point of view, anyway. Though, I can honestly report that every participant I've ever had on this course reported exactly the same sentiment!


[3] I don't believe the term/job-title "Architect" had any currency back then. Whilst it had certainly been invented4 it was certainly not popular. In fact there still was no such Job Title as "Software Designer" back then. My boss had to invent it for me!


[4] I believe that I was one of the (probably many) inventors of "Software Architect"a s a job title. Certainly not unique in that, though! Back in 1989 a technical career-path looked much like "Junior Programmer - Programmer - SeniorProgramer - Junior Systems Analyst - Systems Analyst - Senior Systems Analyst - Business Analyst..." I rejected the whole deal5 stating "Analysis is The Art of Taking Things Apart. I don't want to do that. I want to put things together, and that's called Design (and, at the high-end, Architecture)."


[5] ... in a bi-annual merit-assessment that became (in)famous as the one where I told my Manager, "If you want loyalty, get a dog! This is a business relationship; didn't you understand that?" Needless to say I got no increase that year. ;-)

19 July 2008

Datapro/Vox Spam Wrapup

Its been a long, long time... way too long... since I promised to report back on my complaint to the ISPA in which I accused Datapro/Vox Telecom of being spammers.

The first part of the ISPA's complaint-handling procedure is to have the parties try and resolve matters between them. Datapro's attempt to brush me off was a laugh:

So eventually, somewhere around March, the event wound its way to the ISPA's Complaints Committee who deliberated, and some weeks later, sent me a formal Ruling:

Datapro/Vox Telecom were found guilty of spamming, and fined R100 000 (about EUR10 000 at the time.)

Naturally Datapro appealed this decision, as was their right. Unfortunately the appeals process is quite a slow one, so it was only about 9 June that I was informed of the outcome of the appeals process:

On appeal it was found that Datapro/Vox Telecom were confirmed as being guilty of spamming (through address repurposing.) The fines were, however, reduced to R45 000, most of which has been suspended for 12 months, provided that Datapro/Vox do not spam again within that period of time. Effectively Datapro/Vox have had to pay a paltry R7 500 in fines. Their guilt remains undisputed.

This strikes me as the ISPA having no balls.

The Appeals Committee recomended that Datapro/Vox
...ensure that a working opt-out facility is provided on all marketing emails and to report back to the ISPA complaints administrator within 1 (one) calendar month of receipt of this report on remedial action taken by it. The Appeals Panel recommended (which recommendation is not binding) that such remedial action should also include cleaning its email marketing lists to avoid a repetition of this complaint. This could be achieved by way of a reminder email requesting recipients to confirm their desire to continue receiving such emails (opt in), which is preferable, or by requesting them to indicate their preference not to receive such communications (opt out), which is adequate and then abiding by the indicated preference.
To my knowledge none of this has happened. I have received neither an opt-in request, nor any opportunity to op-out.

The only communication I have had was on 3 June -- 8 days after the Appeal Ruling -- I received yet another marketing spam from Datapro/Vox... still no opt-out link, still no confirmed opt-in. I just have not had the time or energy to follow-up with another complaint of this violation of the suspension conditions imposed by the ISPA on Datapro/Vox.

Further, the ISPA's documentation carries a notice stating
PRIVATE AND CONFIDENTIAL:
All communication is confidential and may not be distributed.
See http://www.ispa.org.za/commsrules for more info.
so I am effectively barred from making the original PDFs available, despite (or perhaps because of) my having made it clear to the ISPA that it was my firm intention to publish the course of events and all documents.

I am also unable to find any notice, link or reference to this ruling on the ISPA website. Are they too afraid of the potential income-loss should one of their largest spammers members get pissed-off enough to depart the Association?

A pretty sorry indication of the general state of the local ISP industry, I'm afraid.

03 July 2008

Power to the Purple

A week of power-supply problems. Not Eskom's fault, this time, but more localised failures.

First the power-supply for the network server had a fan stop turning. I could have taken the chance on the unit working without cooling, since it is relatively lightly loaded -- no graphics cards, only a single disk -- but, since I had a spare power-supply unit handy it was a task of mere minutes to swap the faulty unit out and get the server back into action. It is a fairly key piece of our little home network, being a web-cache, local domain-name server and cache, Subversion repository and file-share space, so we miss it badly when it is down.

Then the power supply on my desktop machine decided to follow suit. Also a fan failure. I hate those crappy little fans! There's absolutely nothing wrong with the basic electronics of the power supply itself, but the ball bearings in the fan have died. Pricing for a new power-supply runs from a little over R100 if I were in Cape Town with easy access to wholesalers, through R200 from a web-shop, all the way to R300 from the local PC shops! This is for the most basic 350W PSU -- none of that fancy gaming-machine stuff for me. (Though I will confess to being tempted by a unit costing around R800, simply because it is alleged to be completely quiet! I'm a self-confessed anti-noise-maniac.)

My guess is I'm going to spend an hour messing about with the soldering iron, installing new fans (I have a couple just lying about) in the "faulty" power-supplies.

At the same time, several warnings from my server-supplier in London telling the story of a week-long tail-of-woe about power-supply into the datacentre. Apparently a failover switch failed to work correctly during a power-outage last Sunday, causing the battery-based UPS to take the entire load for about 10 minutes before the batteries were totally drained. All servers in the DC went down hard. It has taken them until Thursday to isolate the problem and replace the parts (electrical and mechanical) that were at fault.

During the whole affair, all server owners have been kept fully informed via RSS feeds and emails at every step of the way, since there is a risk (however slight) that servers might go down if there is a power-grid outage again and the on-site staff -- now fully briefed on managing a manual switch from grid power to the backup generator -- should get taken-up at just the wrong moment.

This is exactly the sort of thing I expect from server providers and datacentre operators. Everybody understand that, despite the best-laid plans, sometimes shit happens. It is how they respond, and how transparent and communicative they are in responding to the crisis that truly matters.

This is in very sharp contrast to Verizon's datacentre in Durban, where my other client's servers are housed. About 10 days ago they had some electrical work going on in the DC, which in turn made some server-moves necessary. They did all this without warning their clients that there might be some risk to their operations. Needless to say, my client's servers went down without warning in the wee hours of Sunday morning. No heartbeat monitoring in place, so it was Monday before anybody knew that something was wrong. No peep from Verizon to their customers. Half-arsed, I call it.

There's a lesson in all this about Single Points of Failure. I've been warning for over 8 months that having all the servers housed in a single DC, or even in a single city, is a risk. Maybe now the business will take some action, but, given the general lack of respect or attention to the fact that, like it or not, they are a technology business, I have my doubts.

19 June 2008

Damager or Manager?

Since Open Letters to Management seem so flavour du jour, I thought I'd save the following fine rant from oblivion. Names and project details changed to protect the guilty, of course.

DumbTribe is a small startup in the mobile space. They have started seeing some good traction for their product, but are completely chaotic in their "management" of the company. The company is 100% reliant on IT, yet, whilst they're willing to spend an ordleplex of money on fancy new offices, they're astoundingly short of cash when it comes to things like buying another server to act as failover for their single server. Said server is the sole source of income for the business.
What [ManagerX] calls a "blogger tool" is really a form of Content Management system that ends up providing (among other things) Atom and RSS feeds...

[ManagerX] wrote:

> This has certainly taken longer than we initially thought it would. I
> think it was over a few of months back that we were expecting a
> finished blogger tool.

You seem to have forgotten that there were other tasks that YOU prioritised ahead of the "blog" tool -- development of the feed aggregator and the [BigClient] pilot, not to mention system administration tasks more numerous than I can recall, design and coding assistance to my colleague, installation and maintenance of essential technical infrastructure indispensable to organised development (some of which runs on my own servers, at no additional charge to [DumbTribe], simply because that was the fastest way to get the tools in place.) I regret that I am unable to work on more than one thing at a time, but these are rather complex systems dealing with some very erratic, "dirty" data coming-in, and, like most men, I don't multitask well.

> It is of zero use in it's present state(just like the blogger tool you
> created was in it’s 'unskinned' state(to me the level of things that
> fell under 'skinning' was surprising.

First: I warned right from the start that installation of the necessary software was the quick and easy part, but that changing templates -- "skinning" as you call it -- would take at least several days for someone expert in the templating system. I also made it clear that such templating was NOT in my sphere of competence. Evidently nobody was listening to the bits they didn't want to hear.

Second: I will not take responsibility for your inability to produce a coherent specification for the tool. Lack of any technical specification underlies the several misdirections and false starts. A powerpoint does not BEGIN to form a clear technical specification.

Example: I, at one stage, asked you how many "blogs" it is necessary for the system to support. At that time I had in mind to use a particular piece of software as the foundation infrastructure. Your answer to my question was "thousands!" which answer had a significant impact on my technical decision-making, since the tentatively-chosen solution is unsuited to such large volumes. I certainly made it clear that I was unfamiliar with the more suitable tool, and, indeed, the "skinning" -- the writing of custom templates -- turned out to be more problematic than I anticipated. I made a poor guess in the face of inadequate information, a misleading business requirement and insufficient time to evaluate alternative technical solutions.

In the long run it turned out that a "blog" system is just what [DumbTribe] does NOT need. What IS needed is an article/story management system for providing Atom/RSS feed output. This is in the final stages of development.

You seem to have forgotten that using a blog-system was initially mooted merely as a temporary stopgap solution to provide a mechanism for getting article content into feeds; it was never intended to be the "real" solution. What I have been developing is such solution. Assuming you don't sabotage delivery with yet another interruption.

I would have been finished 10 days ago had [my colleague] had enough spare hours to assist me on areas where I do not have the deep knowledge of data structures and code she already has in place, and if the "specification" had not been changed on a number of occasions. Unfortunately other more pressing issues have had to take precedence on her time, with resulting delays on the "blog" project. Furthermore the assistance I have (gladly) given [said colleague] on other projects has also had the effect of taking several days from "blog" development.

> I am sure you understand what this looks like from our end. It just
> feels like you can’t give us what we need.

Yes, I am pretty sure I DO understand. It seems to me that you think one of two things; either:

1. That I am incompetent to produce working software, or

2. That I am dishonest and lie to you about my activities.

I am neither, and find either accusation hurtful, denigrating, and completely unprofessional. Software development, unlike so many other jobs, does not allow one to delude oneself about the limits of ones knowledge or abilities, so the charge of incompetence is easier for me to dismiss when I consider its source; I know exactly how good I am.

Clearly you have absolutely no clue how software development works, nor what is a "normal" pace of production for software systems. The fact that your most-recent experience of software development is exemplified by [colleague], who is prepared, for reasons I cannot comprehend, to endanger her health and wellbeing by working outrageous hours in order to meet ridiculous, unrealistic and arbitrary deadlines does not alter the truth of what I am saying. Nor is it my place to attempt to teach you how software development works; for that sort of work I charge considerably more than you pay me.

The fact that you have badly under-resourced this area of the business is hardly my fault.

> I am so frustrated and feel if I have to explain what we need again I
> will go mad.

Unfortunately software is all about the detail. If you do not tell a developer all the detail that they need, they will guess, and likely guess wrong.

Therefore, where details are lacking I will ask again and again and again. I have on occasion asked users to describe their requirement from beginning to end as many as 8 times in a single day in order to be sure I understood the requirement. Then I asked them a couple more times the next day. I am deeply sorry if my need to know what you want in full detail drives you mad -- I certainly do not wish to cause such mental anguish.

> Please can you confirm that you understand and accept all the
> functionality that we need

No. I do not believe I understand what you need, particularly as you keep changing the requirements. I am not a mind-reader, and you have not produced a comprehensive technical specification.

Example: Your comment on Monday, "Make sure we can direct the 'see original story' link to a site of our choosing (e.g. [ClientA] sites or [ClientB] sites)" This directly CONTRADICTS the requirement laid out in your powerpoint that NO such link be present. What am I supposed to do with that? I can put such a link in (though linking to what, I have no idea, nor do you say -- another missing detail) or I can leave it out (as is done at present.) I am happy to do either, or to make any changes you require, since I understand that business requirements can and do change from time to time. However, changing what is already implemented does unfortunately take some time and cannot simply be done with a wave of a magic wand.

Example: The original requirement was for articles to have a single image attached. Then an image or a URL. Then multiple images or URLS. Then it became "unusable unless we can upload video". Then we didn't need video any longer. Then we were back to one image/URL. Currently I am informed that multiple images/URLS are a non-negotiable requirement. Every time a change such as this is introduced it costs me hours or days in the attempt to comply.

And you wonder why there have been delays.

My current aim is to get the system in place with the capability to upload ONE image or attach ONE URL, either of which shall appear at the tail-end of the feed content (another detail not specified.) It is my belief that it is better to get SOMETHING up and running, even though we all agree that it is NOT the end-product desired. Then we evolve it to the state we desire. After all, that is why it is called "soft"-ware.

> and let me know what *date* we can expect a working tool to start
> testing.

In the absence of a full, clear, comprehensive specification, no such estimate can be made by anybody. In effect you are waving your hands about, saying "build me a Tudor-style house over there" and then demanding that I tell you how long it is going to take without giving me the plans for the house, specifying the building materials, size of the house, number of bedrooms, etc. When you supply me with a proper User Requirement Specification -- for which I will gladly supply a Word template outlining all the necessary information it should contain -- I may consider beginning to make estimates.

> I don’t think it is unreasonable for me to ask you to commit to a
> deadline, the brief is surely clear now after all these emails back
> and forth and the very easily accessed example of blogger.com which I
> asked you to use as a starting point.

On the contrary I think it thoroughly unreasonable to make such demands. The "brief" (whatever THAT is) is non-existent. To point to blogger.com and claim that that is what you want is ridiculous, since blogger.com is totally unsuited to your needs. If it were suitable there would have been no need to build anything else and we would not be having this conversation.

I will remind you that I am contracted to deliver 40 to 80 hours per MONTH of work -- not per week. This was deliberately and clearly negotiated up front. Consequently I do not work full 8-hour days on DumbTribe activities, which, too has its effect on delivery schedules. The fact that I consistently seem to end up working more than the agreed number of hours per month seems to be taken for granted, or alternatively is regarded by you as an attempt to rip you off. On the contrary, it is a good-faith attempt to come some way further than I strictly, am contractually obliged, in an attempt to help DumbTribe meet its goals.

A lack of planning on your part does not constitute a crisis on my part.

In the extremely unlikely event that I elect to extend/renew my contract, should DumbTribe wish to do so, please be assured that I will require a considerable tightening-up of the conditions relating to all of these issues.

13 June 2008

Otherwise Occupied

Spending my time preparing for a course -- "Introduction to Java Programming" -- for a Major Corp in CT next week. It's a course I must have presented dozens of time. Perhaps hundreds. But its been about 5 or 6 years. Actually, I did teach it once a year or two ago... I'm sure the company I did the work for were satisfied. I'm pretty sure my students were mostly happy. I wasn't. The break from teaching had left me rusty. God knows how many tiny but important details I left out by mistake. Did I talk about volatile variables?

In a 5-day "intro" course, there's not much time to fuck about. There's a certain set of Java's features that you simply have to talk about, at least in a simplified and superficial way, simply because your students are guaranteed to trip across them almost immediately they walk out of the classroom.

On the other hand -- let's be honest -- there's a limit to what the human brain can absorb in 5 days!

This time around its a little different. I'm teaching for a single company. The Program Manager's focus seems to be quite squarely on getting her people up-to-speed, and not on any other corporate ass-covering or thinly-veiled-reward-holiday-for-week sort of bullshit. Believe me, I've seen that crap all too often!

I was able to stretch the course schedule to 6 days instead of 5 -- a huge relief and I can take things at a slightly less frantic pace. I frankly don't quite know how I would fit the basics into a 5-day course anymore, given that one has to cover (at least at a shallow level) enums, annotations and generics. I guess that maybe the modules on threading and concurrency would have to fall away, but how the hell do you justify doing that when you absolutely know that your students are -- willingly or un- -- going to confronted with those as soon as you can say "Boo"?

Well, I've been working hard at getting my teach up to scratch, updating the (Day 2) OO module to include enums, and the (Day 4) module on the Collections API to teach generics, etc., etc. Of course my (development) contract customer is a little miffed that I'm not putting in 18 hour days for them... Tough Tittie -- their time is worth about 1/4 what I'll earn from teaching, and they've hardly made any effort to improve the disgusting working conditions!

I think it will be a killer course. I'm certainly aiming to make it so!

Along the way, Jason and I have been having some very entertaining discussion about the deeper, occult details of Java concurrency and system design. Something clicked last night; we both reached the conclusion that it would be fun (and probably educational for all concerned) to get more people in on that conversation, so we've revived my idea-of-many-years-standing -- a workshop/seminar format gathering, aimed at top-level, ten-years-plus-experience OO designers, architects and senior developers: A general framework to guide and channel discussion, aiming for an honest sharing experiences and learning. No fluff!

I facilitated (I would hardly dare claim "taught") one such many years ago, and, in my own humble opinion, it was the best course I ever led! I think that quite a few of the participants would agree with me. Certainly a majority of them kept in touch with me for an otherwise inexcusable number of years afterwards! After all this time, there still exists a huge gap in the professional-education space... beyond a certain level, expert practitioners (and I think it is true of any field, not just software design) have nobody to talk to -- nobody around them who is at a level where they can meaningfully push back on ideas -- nobody to brainstorm with. Or at least, far too few. The Advanced Practitioner Workshops would aim to try and (partially) fill that vacuum.

So maybe its back to teaching again for a while! It could be fun to focus on actually delivering actual deep learning for a change instead of just aiming to be Brilliant Clown For A Week!

03 June 2008

UI Design Encourages Mistakes, Boosts Profits

Having just finished my banking and tax admin for the month, I fire up the bank's online system to fork the money over to the various landsharks, fatcats and leeches.

Bank payment UITo make the various tax payments I first have to select which of my accounts to use for paying.  Then I select who I want to pay.  (For some value of "want".) The system insists that you click the "Search" button to check that you have entered the corect payee account details, but, of course, it only tells you this after you navigate away to the next page, and, when you return to the payment details form, it throws away all the details you have already captured.

I also have to enter a 19-digit reference number and, guess what, if I get it wrong (as I am likely to do with such a long number) it only tells me on the next page, and, again, throws away all the work I have done to fill in the form, forcing me to redo it from scratch.  Including forcing me to redo the payee "Search", despite the fact that my browser has captured the field details perfectly.

The real kicker is that it also ditches the account-number from which I want to make payment, substituting the "default" account (which happens to be my personal account and not the business account)  Of course, the account number is metres away up at the top of the web-page, so I don't notice that I'm making payment from the wrong account.

As a result I pay business taxes out of my personal account.  When I eventually discover my mistake, I have to transfer the money over from my business account to fix things.  And I am going to get hit with the withdrawal fees again.  To add insult to injury, the payment pushes my personal account into overdraft.  Bam!  Overdraft fees!

This UI is so wrong, in so many way, you would think that the bank  would have "inspired" and "motivated" to fix this monstrosity years ago.  Do you wonder why Standard Bank can't be "bothered" to fix their broken user-interface?  Do you wonder why they make incredible profits?

Did you think the two facts are unrelated?

20 May 2008

Invisible Work

Here's a great quote from Jim Waldo, courtesy of Dan Creswell's Blitz blog:
…Even worse than not being visible to the customer, work done on designing the system is not visible to the management of the company that is developing the system. Even though managers will pay lip service to the teaching of The Mythical Man Month, there is still the worry that engineers who aren’t producing code are not doing anything useful. While there are few companies that explicitly measure productivity in lines-of-code per week, there is still pressure to produce something that can be seen. The notion that design can take weeks or months and that during that time little or no code will be written is hard to sell to managers. Harder still is selling the notion that any code that does get written will be thrown away, which often appears to be regression rather than progress.
Never a truer word!

08 May 2008

Reprise

My week for being haunted by old ghosts.

A bit more than a week, actually... It all started last Thursday with a call from a lady working with a lot of organisations that support HIV counselling, treatment and management. A lot of organisations. She had tripped (how?) across a product/project that a few of us put together some years ago that we called Projectory. Projectory is a collaboration and communication platform, specifically aimed at software-development organisations and teams. Think of CollabNet. But better, of course! ;-) Certainly quite different in some key ways! Except we never got the business off the ground, mostly through an unlucky turn of events that resulted in us losing key sales people at a most critical time.

My caller was wondering whether the Projectory platform could be adapted to help them to communicate, collaborate and coordinate better with a couple of hundred other organisations. Well, we've set up a meeting for next week, and we'll see... What a blast from the past, though! I had all-but-forgotten about Projectory... Thankfully I have the code archived away somewhere safe.

And then it happened again. A call from an ex-colleague a couple of days ago: Could we put together a rough estimate and proposal for a social-networking platform for World Cup 2010. In case you're living in a cave (or the USA where "football" means something completely weird) South Africa will be hosting the 2010 Soccer World Cup, and, for South Africans, it is a very big deal. This is, after all, the second biggest sport event in the world after the Olympics. (China get the hell out of Tibet!) The weird bit is that what's being asked for is very, very close to the project we called Flightwish -- a social-networking platform centred around group travel opportunities and concepts. We failed to get funding for Flightwish, though we tried hard. Normally I would favour a small-start, organic-growth, little-or-no-funding startup model, but that path is clearly a very poor fit for a Grow-Big-Fast webalicious venture. Since then, a few others have started playing in that space, but I have yet to see any of them put together the exact combination of ingredients we planned. Would we have done better? Who knows?

But now we may just get a chance to try it again.

Key takeaways:
  • I suck at Sales and driving Sales, therefore I am a poor fit for CEO of a startup.
  • Flightwish taught me a deep hostility to the idea of a single "window of opportunity"; it's rubbish.
  • Beware the Websites Of Yesteryear! You put up a website. It's out there. You forget about it. Google doesn't! Fix them up or shut them down.
  • The "social" potential -- the ways that the web opens-up for collaboration and group communication -- we've barely scratched the surface of what's possible.
  • VC people in SA are mostly bankers with fancier job titles. And we all know the collective noun for bankers, don't we...

30 April 2008

Software Estimation Considered Harmful

problem

"So how long do you think it's going to take?"

We've all been asked the question a thousand times and more. Project Managers, Client Liaison, Salespeople, Marketing Managers,... they all want to know. And we, like sheep, like the suckers we are, because we try to please (you try, too, please!) suck hard on our spacebar-calloused thumbs, and guess.

"I guess a couple of days."

"A week."

"Three months."

"A year; maybe 14 months."

Nobody really trusts those really long guesses, though, so that's where the project management experts get involved, break the task down into itty, tiny bitty little bits, parallelise them, ALAP, ASAP, lead and lag. And then we all sit down around a table, and for each and every one of the itty tiny bitty bits the project manager asks the dreaded question.

"So how long do you think that one's going to take?"

The Real Answer, the Truest Truth, is, "I don't know." Perhaps a tiny voice deep inside our soul cries out, "I don't care! It is going to take as long as it takes." But for mysterious reasons all tangled up in our wetware, all tied up in the social hierarchy dynamics of the human ape and those twisty strand of ribonucleic acid in our hardware, "I don't know" marks me as less-than-competent. And "I don't care" is career limiting; "Not a team player. Fails to identify with the organisations goals and ethos."

So we guess, and we guess, and we guess again. And we're (almost) always wrong!

For 50 years we, as a craft, as an industry, have been guessing wrong. For 50 years our projects have mostly finished "late", and we keep wondering "Why?" Books have been written, Methodologies developed, PhDs awarded, Management Disciplines imposed and entire Consulting Industries built on the premise that it is ''possible'' to estimate software effort better than we presently do. Or, if we can't estimate better, then we can manage the estimation risk better. Or the process. Or those pesky damn programmers.

All in vain. The projects keep coming in late.

where it all comes from

How did this abysmal state come about?

I think it is rooted back in the 1950's, when the first Big Software projects were undertaken, most notably by the US military. Quite naturally they applied the project management strategies that have worked for them since time beyond time. Strategies that have successfully built forts, dug moats, laid siege to cities and moved large numbers of soldiers across continents and seas. The trouble is that those are extremely well understood problems that have been solved thousands -- millions -- of times, ever since Akkad invaded Ur. For such classes of problems, estimates of time and effort are pretty reliable. If you're a logistics planner, an army engineer, you know, from loads of practice and handbooks distilling five thousand years of experience, just how long it takes for a boatload of soldiers to move a given distance, and just how much food, water and fuel they need along the way.

"Where there are in the field a thousand swift chariots, ten thousand heavy chariots, and a hundred thousand mail-clad soldiers, with provisions enough to carry them a thousand ''li'', the expenditure at home, including entertainment of guests, small items such as glue and paint, and sums spent on chariots and armour, will reach the total of a thousand ounces of silver per day."
-- Sun Tzu, ''The Art of War''
It is an extremely well understood problem domain!

Software development is a different sort of beast. It's all new to us. Amost every problem has never been tackled in its entirety before. I don't claim that software development is unique in this; I am pretty sure that developers of new aircraft, ship engines, new forms of bridges, all face the same problems as software development. What -- perhaps -- distinguishes software development is that it's all new every time! We never get to repeat our past developments. If we were repeating -- exactly -- repeat-developing a requirement -- in every detail -- we would already have the code and we'd have no need to write anything! And if there's one thing I've learned about developers, its that we -- most of us -- hate doing the same thing twice! It's probably a function of our predisposition towards ADD/ADHD.

all change

But, no! In truth there are always differences between the Systems That Have Gone Before, and the Systems We're Developing Now. Even if you've developed interface to a dozen payment gateways, 100 gets you 1 that the next payment gateway has some unique characteristics all its own. Or perhaps some key technology has undergone a significant change in the last few years. Or some new tech has crept into the picture. "Our service is provided through a RESTful API"...

Then, too, how do you account for the Buggerance Factor. Murphy's Law. The simple fact that even the simplest piece of software depends on a ''huge'' number of other bits'n'pieces being in exactly the right places, configured exactly right, at exactly the right time?

I should not have had to spend an hour this morning tracking down the fact that a key library -- a jarfile that should always exist in a standard, accessible place -- had mysteriously been Taken Up. Vaporised. Gone. Perhaps my disk is going flaky? It shouldn't have happened. I should not have wasted an hour figuring out why the application wouldn't run. But I did.

I should not have spent a morning last week discovering that, despite the Vendor's persuasive assurances to the contrary, the version 8.2 driver does emphatically not work properly with the version 8.1 server. Let us not even ask the question, "Who installed the 8.2 driver?" A fruitless waste of time, energy and stress hormones.

How the hell do you predict that particular morning and figure it into your time and effort estimate? Or the half-day spent figuring out that there's a bug in a key data-access library you're using (and it's not your choice that mandated its use, but some arbitrary "policy".) And then another hour figuring out a way to work around the bug. Just how exactly, when you're Project Planning some 4 months ahead of a frustrating and unproductive morning, do you predict those?

You just blew your estimate.

managing estimation risk

I am well aware of various approaches to software development that try to futz around the problem -- some of them with some marginal success -- by giving up the idea of a project being "finished". But nobody seems willing to confront the central problem head-on.

Estimating "How Long It Will Take" is a Broken Idea.

Like the drunk looking for his spectacles under the streetlight "Because that's where I can see to look for them" we keep searching for ways to make estimation more accurate, more reliable, more amenable to conventional management thinking. What we really need to do is screw our courage to the sticking point, and accept that there is, really, honestly, truly, no alternative:

We must completely abandon the whole concept that software effort is amenable to estimation at all.

call to arms

Give up the crutch!

The next time -- and every time after that -- that someone asks you "So how long do you figure that's going to take?" -- "So when do you think we can go live?" -- Just Say No!

Just say, "It will take as long as it takes."

I guarantee you some excitement in the short minutes immediately following, but let go of your fear! Immediately you will find yourself skulling in the calm pond of assurance and truth that lies beyond the fear. Live and enjoy this Truth, for it will set you free. If other people want and need to make deadline commitments, let them be the ones to suck their thumbs, making up fantasies and lies. Do not allow them to push that responsibility onto you. Don't allow them to turn you into the liar. Just tell them,

It takes as long as it takes. It always does, anyway!

25 April 2008

Quotable

"The Ark was built by one man.
The Titanic was built by a team of professionals."

20 April 2008

Wizzards

Programming software is Magic.

I don't mean that there's something mystical about it, nor that it is intrinsically inaccessible to ordinary people. Nor (I emphatically add) do mean that it is like Magic. In every aspect I can think of, the act of Programming software meets all the criteria for performing Magic. Magic in the Swords and Sorcerers or Unseen University sense. "Alakazam!" and the Prince turns into a Frog. "Shazam!" and you're whisked away to a far, far place at high speed.

Just look at the facts: We (programmers) write programmes -- spells -- in arcane and cryptic symbol languages unknown to the common mob.  Get the slightest part of the spell wrong, and, at best, it fails utterly to do anything.  At worst it runs amok and fearful consequences ensue -- fires, floods, loss of money and even life!  Get it just right, in every teensy, tiny, ball-aching, nit-picking detail2, and Lo! out of nothing, stuff happens in the real world.  Gold changes hands.  Trains run on schedule.  Music plays and Feyries dance1.

Where nothing was before the spell was cast, something comes about solely because of the spell.  That's Magic.

And, just like in the fantasies where Wizards keep pet Dragons and dribbly candles set atop skulls are the acme of interior decoration, programmers frequently work at odd hours, with intense, monomanic concentration bordering on the inhuman.  And, like the traditional Wizardly Schools, programmers are admitted to different schools of various arts and degrees.  So we have Clerics -- programmers content to churn out the boilerplate code needed to keep the wheels of commerce (and most web applications) running, but lacking any true proficiency with martial weapons of higher degree; Monks -- who eschew the use of particular weaponry but, ninja-style, willingly embrace whatever comes to hand as combat fodder; Wizards -- capable of serious Magic, but forget their spells once cast, capable of wonderful stuff, but doomed to repeat it -- with minor variations -- time after time; and then there are the Sourcerers -- Masters of The Source3 whose code is so elegant and expressive, so parsimonious and pretty as to make brave Programmers weep with envy and admiration.

No, you can not deny.  Programming really is the realisation of the ancient idea of Magic.  Say the Magic Spell and Change Reality.

[1] Well, anybody who wants to dance, I suppose, really.
[2] ...and My God, there's an inordinate amount of crappy detail that all has to be Just So!
[3] Waves to Ken, Doug, Dave, Jason, Paul, Johan, Bob, Brian, John and several dozen others...

15 April 2008

Bad. Punctuation,

Tripped across this little quote on /. this morning:
A Linux machine! Because a 486 is a terrible thing to waste! 
-- Joe Sloan, jjs@wintermute.ucr.edu
Alas, I can clearly see that somebody screwed-up the punctuation.  Surely that should read
A Linux machine! Because a 486 is a terrible thing. To waste!

26 March 2008

SABC's website sucks

Email to info@SABC.co.za:
"Why does SABC's website suck so badly when viewed in Firefox?

"Not to mention that it is completely unusable with Javascript disabled, which renders it inaccessible to people using Braille readers or text-based browsers of any kind; this violates the constitutions provisions against discrimination."
BTW: if you leave off the "www." prefix, you get to see exaclty what software they're using to drive their (very b0rked) portal.  Now I'm not suggesting that this might render them susceptible to getting the portal cracked, but anybody who has set up a portal server that incompetently has quite possibly left some default logins/passwords in place.  Maybe?

Not that I'm suggesting anything, mind...

01 March 2008

User-interface Reboot

This article by Mr Mirchandani gets it exactly right: UI again ...don't pretty up, destroy!

I have never forgotten the experience of early last year. Our car had been stolen, and we were jumping through the licensing department's hoops to get the old car de-registered, and our new car registered.  Well, 10-year-old, 2nd-hand car, since that's all we could afford with what the insurance company deigned to pay out -- another saga for another day.

First we could not de-register the old car, because it was flagged on the licensing system as "stolen", so no changes to its details are permitted.  WTF?  We could not unflag it, since that would require the police to mark the car as recovered, complete with verification of engine, chassis, VIN and registration numbers.  Eventually we left the matter in the hands of one supervisor who took pity on us as I crumpled in the face of this actively-hostile "information" system.  She solved the impasse by going outside the system: phone calls to a special contact in Pretoria -- "high friends in low places."

Then we had to register the new car.  The details had to get captured no less than 5 times!  Twice, manually by myself, the remainder by the clerk punching a terminal.  And two of those instance involved recapturing the vehicle details from a form still-hot from their system's laser printer.  The system already had the details, yet they still had to be manually recaptured.  This is insane!  Weren't computers supposed to save us work?

23 February 2008

As BAD as Some can be, Others can be GREAT

We interrupt the on-going diatribe between my self and Datapro/Vox Telecom[1] to bring you Good News for Modern Persons.

In the supermortal words of Hubert Farnsworth, "Good News, Everyone!".

Last eve some mishap caused my DSL model/router to disconnect.  For some while it failed to reconnect: AUTH_FAIL, it said.

Now, my ISP, WebAfrica, whom I hold in very high regard, has been having an occasional little trouble in recent times with their authentication servers.  So: patience is the order of the day.  It was quite late in the day, so my bed called, nothing in my little local network really needed Internet access overnight, so I left matters until the morning, in the hopes that the problems would be resolved without any input on my part.

They were not.

So... Onto the phone this morning.  Less than two rings!  (Contrast this with giving up after an hour on hold last week with Telkom!)  Spoke to a chap who was remarkably candid: "Yes, we have had a problem, and a few accounts seem (for reasons we don't fully understand, yet) to have been stuck in an "inactive" queue.  We're terribly sorry.  I am sorting it out right now [clickety clickety clickety click]; would you like to hold?"

I declined to hold.  The pain of being on hold to Telkom being too fresh in my psyche, I suppose.  After suitable pleasantries I hung up.

A couple minutes later the phone rang.  Same chap from WebAfrica.  " I see that your modem seems to be having some trouble connecting.  Could we please confirm the password it is using to connect...?"

Well, Bugger Me Sideways With A Spoon!  Not only did WebAfrica's support guy sort the problem out instantly, with an ordinary, human-to-human acknowledgment that something had, indeed, gone wrong, but, after I had explicitly said "Ticket closed; I'll call you if there is any further problem." had monitored the situation to make sure that I -- The Lowly Customer -- had been properly sorted out, and called me back to make sure of it!

What am I saying, here?
  • I could have raved about not being kept on hold in some support-queue for an hour or longer.
  • I could have raved about the great service I received during the handling of my support call.
  • I could have raved all night in San Fransisco with the hot chick on her way to Hawaii (but that's another story!)
This guy -- unasked for -- stayed attentive to my little problem until he was as sure as he could be, that it had been solved to my satisfaction.  Not his.  Not Webafrica's.  Mine.

Here's a significant point: None of us (modulo the Absolutely Bloody Minded) is so stupid as to believe that everything Works Flawlessly All the Time.  Shit Happens.  We know this.  When it does, please don't lie to us and use phrasing designed to imply that we, the Customer, are Stupid, Insane and/or Lying!  Please don't pretend that it is Somebody Else's Fault or an Act Of (somebody's) God. (Hello, Telkom!)  If you've fucked up, admit it, apologise, and move on. Nobody will hold it against you.  In fact, given the current climate of Assumed Corporate Infalibilty, we'll sympathise and likely offer to help you fix it!

Just say "Yes.  We Had a problem.  We've fixed it. (OR: Here's what we're busy doing to Fix It.)  We're sorry."

If it is a significant proportion of the working day, offer a credit for the lost service time.  Not difficult, is it?  Not Rocket Science!
I cannot think of a way to praise this enough!

This most recent incident is the perfect exemplar of the sort of brilliant, attentive, honest service  I have unfailingly received from WebAfrica!  I have had a friend[2] phone me up especially to say, "Thank you for putting me on to WebAfrica as a service provider!  I've since recommended them to at least 15 other people!".

I kid you not!

If anyone in South Africa wants or needs ADSL service, Internet access or web-hosting, do yourself a favour: www.webafrica.co.za

Their rates are amongst the lowest around.  Their service is out of all proportion to what you pay!  (i.e. It's brilliant!) If they ever get bought out by Vox Telecom I shall probably have to leave the country -- and even then I won't find an ISP as good!

[1] A "keyboard/finger" interaction nearly made that "Pox Telecom", whIch would have been appropriate...


[2] We've known each other over 35 years, now... I think that qualifies as friendship, no?

22 February 2008

Taking on the Spammers: Datapro/Vox Telecom - Part 4

Mr Douglas Reed, CEO of Vox Spamacom Telecom, parent company to Datapro, replies:
We run an ISP with over 18,000 corporate customers and 180,000 SME's and
we have customers who utilise various services.  These include list
servers where customers use their own databases and we don't have full
control.  The DataPro and Vox databases are within our control and
consist of individuals and organisations who have provided their details
to us. The reason we have you on our Company database is because you are
obviously listed as a technical contact for some of our customers.  We
cannot offer opt in opt out facilities for our communication to our base
because the news letters communicate important information that the
technical contacts need to be aware of.  However if you want to be
excluded please give us the details and provide us with new technical
contact details.

The other choice is do what the rest of us do and add the user to your
junk mail list.

Interestingly enough this mail ended up in my junk mail folder which
basically means that I received unsolicited mail from this in the past
or you cc'd thousands of people.

Is it just me, or does this sound just a tad arrogant?  What I am hearing: "We're big; that means we can spam with impunity, since we're too big to get blocked." and "Shut up and eat your spam!"

My response:

Dear Mr Reed, On 18/02/2008, Douglas Reed <douglasr@datapro.co.za> wrote:

> The DataPro and Vox databases are within our control and
>  consist of individuals and organisations who have provided their details
>  to us. The reason we have you on our Company database is because you are
>  obviously listed as a technical contact for some of our customers.

The (many) spam emails that form the basis of my complaint to ISPA are directly from Datapro and Vox Telecom; this is not about spam from your customers.  One spam message bears your name as "signatory".

You will note from my earlier correspondence with Maggie Cubitt that I have tried repeatedly, using numerous channels, to "opt out" of these mailing lists, without any success.

Why don't your opt-out procedures work? (As required by the ECT Act.)

Although some of your technical staff are certainly in possession of my email address as "technical contact" for some of our mutual customers, this does NOT extend a license to your companies to send me unsolicited bulk email on ANY subject.

Further, I will note that I have never -- not even once -- receive a bulk message on any technical subject.  The emails forming the basis of my complaint have ALL been of a nature that can only be characterised as "marketing crap".  I did not, ever,  at any stage, give any person or system representing your companies, permission to send me marketing crap.  The fact the your companies have done so is known in the email management industry as "address repurposing" and is considered a sure sign of "spam spoor".

>  The other choice is do what the rest of us do and add the user to your
>  junk mail list. I will repeat what I wrote to Ms Cubbit:

<quote>
having my own email address removed from your mailing lists is of only limited interest to me in this matter.  The larger issue, which it is my main purpose to tackle, is that of Datapro and Vox Telecom blithely spamming, over an extended period of time, continuing in the face of numerous good-faith attempts to unsubscribe, and in direct violation of

1) their own Terms of Service,

2) the email provisions of the ECT Act, and

3) the ISPA's Code of Conduct.
</quote>

The point this: "adding Datapro/Vox Telecom" to my "junk mail list," as you suggest, fails to eliminate or mitigate the primary complaint against spam: the receiver has to pay for it. Putting Datapro/Vox Telecom into my "junk mail list" does not mean that Datapro/Vox Telecom cease being spammers.

To (attempt to) be completely clear on this: since you seem to have overlooked the point:

* This is not about Datapro and Vox Telecom spamming ME.

* This IS about Datapro/Vox Telecom spamming AT ALL.

>  Interestingly enough this mail ended up in my junk mail folder which
>  basically means that I received unsolicited mail from this in the past
>  or you cc'd thousands of people.

Nonsense.

No such conclusion can be inferred.

Having personally administered email and spam-filtering  systems, I can tell you that you cannot draw any such conclusion; thogh it /may/ call into question the competence of the people managing your spam-filtering systems.

>  We run an ISP with over 18,000 corporate customers and 180,000 SME's and
>  we have customers who utilise various services.  These include list
>  servers where customers use their own databases and we don't have full
>  control.

What I read into this is that you believe that your organisations are "too large for the rules to apply".  I have some bad news... There are other organisations far, FAR larger that manage to adequately, and to the full satisfaction  of the anti-spam community, police their customers' mailing lists and email activities.  I am pretty sure that both Outblaze and AOL are larger than your operations; both manage to maintain an impeccable reputation for managing the spam problem and speedily terminating spammy customers.

Of course, neither one spams their customers directly, as your organisations have done.

Part of your (companies') responsibility to the Internet community is to police your customers and their mailing lists.  Ways to do this include monitoring their behaviour, and maintaining and ENFORCING uncompromising Terms of Service.  Your response suggest an unwillingness to do so.  This is a slippery slope.  Next your sales-staff will be writing "pink contracts". (Google for it!)  Should you require access to better expertise than your organisations evidently possess, I shall be glad to forward my consulting rates.

All of this remains (largely) irrelevant.  The numerous spam messages I have received are from your organisations; not from your customers. Your unwillingness to eliminate spam from /within/ is, perhaps, indicative of your willingness to tolerate/profit-from spammy customers from without.

Here is the response I expect: As I see it (prove me wrong?), you have two choices:

1.  Throw away all mailing lists under your control, and start from scratch to build new mailing lists.  Of course you WILL follow established Internet procedures for building permission-base email lists. (Somehow, I doubt this one...)

OR

2.  Send a ONE TIME email to all addresses on your mailing lists, explaining (in full) the situation, expressing your companies' regret that such an unacceptable and untenable situation has come about through the action of a few misguided individuals, and asking the recipients to confirm that they WISH to be subscribed to the relevant mailing list.  Should recipients so confirm their desire to participate, your staff should proceed in the full confidence that those persons have positively opted-IN.  Any email address that fails to reply, or that expresses a desire to opt-OUT must be removed from your databases.

This (second) option should be followed-up with a comprehensive on-going (so that new-hi[r]es get the message, too) educational message from the organisation: "We don't tolerate spam in any shape, manner or form." (together with a detailed explanation of just what that means.)  Your marketing and sales staff may require particularly persistent education.

Forgive my lack of optimism.

Since you (read: your organisations) do not know the email address(es) being spammed, you may be sure that I am in a position to monitor your organisations' actions on this, and will report accordingly.

PS:  You, Mr Reed, might wish to consider that a small one-man consultancy such as myself, may frequently be in a position to make recommendations to customers concerning their choice  of service providers in the Internet Services industry.  Either to recommend providers, or, alternatively, to discourage use of any particular provider.  Your call...

If anybody out there thinks I am being irresponsible or unreasonable (obviously with the exception of any Datapro or Vox Telecom employees or agents!) please, please say so by leaving a comment on this blog.  I promise not to delete any relevant comments...

Let's just note for the record that he failed, completely, to address any single point of substance or question in my response...

16 February 2008

Taking on the Spammers: Datapro/Vox Telecom - Part 3 - email Ping Pong

Obviously the spammers thought they could just listwash me and be done.  Here's Datapro's latest response:
Subject: RE: Response to ISPA complaint
Date: Fri, 15 Feb 2008 12:55:45 +0200
From: "Maggie Cubitt" <maggiec@voxtelecom.co.za>
To: "Mike Morris" <me>

Hi Mike Apologies, as I can fully understand your frustration, which is why I am attempting to resolve it comprehensively and finally. I am unable to find the e-mail address mikro2nd@gmail.com on the Contacts database from the DataPro CRM.. and you have extracted the delivery address from your notepad doc. Can I please just confirm that the Newsletter was delivered to the e-mail address mikro2nd@gmail.com?
My response to them:
Maggie Cubitt wrote:

> Apologies, as I can fully understand your frustration, which is why I
> am attempting to resolve it comprehensively and finally.

Please understand that having my own email address removed from your mailing lists is of only limited interest to me in this matter.  The larger issue, which it is my main purpose to tackle, is that of Datapro and Vox Telecom blithely spamming, over an extended period of time, continuing in the face of numerous good-faith attempts to unsubscribe, and in direct violation of

  1) their own Terms of Service,
  2) the email provisions of the ECT Act, and
  3) the ISPA's Code of Conduct.

> I am unable to find the e-mail address mikro2nd@gmail.com on the
> Contacts database from the DataPro CRM.. and you have extracted the
> delivery address from your notepad doc. Can I please just confirm that
> the Newsletter was delivered to the e-mail address mikro2nd@gmail.com?

The spam was not delivered to that email address, but another one.

I am not willing to assist you in listwashing -- the much-loathed practise whereby spammers remove the addresses of the whiners, but continue to blast their unwanted spew out to the Silent Majority Who Just Hit Delete.

I never opted-in to any mailing list belonging to Datapro or Vox Telecom, but was placed on it without my knowledge or consent via person(s) with whom I had contact for purely technical purposes on behalf of my own clients.  This, in turn, means that my email address was repurposed for marketing spam.  In turn Datapro's mailing list was repurposed by Vox Telecom, a company with which I have certainly never had any business relationship.  (Yes, I do understand the relationship between the companies.  No explanation needed.)  Please take note that this is NOT the only list from which I get spammed by Datapro, so your problems are deeper and wider than listwashing a single whiny anti-spam "activist" from a single ill-constructed mailing list or database.

If your lists are NOT fully confirmed-opt-in (and clearly they are not,otherwise I wouldn't be bothering you), then they're spammy lists until you can verify, with a full audit trail, that each and every recipient has positively confirmed their wish to opt in.  Any addresses that cannot be so confirmed must be removed from your databases.  All databases.

The procedure for confirming mailing-list opt-in has been well-established, well-understood, standard practise in legitimate email management for at least the last 30 years, and is correctly implemented by every respectable mailing-list management system.  I would expect an ISP as large as Datapro to be conversant with such established, accepted, and widely-implemented industry-standard, and to have the resources to ensure compliance.  I realise that these practices are somewhat more stringent than required by SA law, but will point out that the ISPA Code of Conduct (para 28) mandates that "ISPA members must operate with due regard for established Internet best practices, as set out in the various request for comment (RFC) documents and as mandated from time to time by established and respected Internet governance structures."  That reads: "established Internet best practices", not "ineffective South African law".  I believe that mailing list operation is covered by RFC-3098 among other resources.

Furthermore, you will, no doubt, have noted that the sample email sent to you is in violation of even the very modest requirements of the ECT Act.  Not to mention the long-term on-going failure to heed good-faith removal instructions as required by the Act.

I trust that Datapro's forthcoming response to this will measure up to the full scope of the organisation's evident ignorance of, or unwillingness to implement, Internet standards and best practise.
Forgive me my skepticism... ;-)

Taking on the Spammers: Datapro/Vox Telecom - Part 2

Yesterday, 14 Feb, the following response to my complaint to the Internet Service Providers' Association about one of their member's spamming activities:

FYI: Mr Reed is the CEO of Vox Telecom (the parent company), so hopefully we've got the attention of a Big Shot.
From: "Maggie Cubitt" <maggiec@voxtelecom.co.za>
To: <me>
Cc: "Douglas Reed" <douglasr@datapro.co.za>

Hi Mike As a listed Telecommunications Company we do take any reports of this
nature extremely seriously. We were very concerned to receive the
notification of your complaint to ISPA, and are obviously anxious to get
this resolved as a matter of urgency.

As there are many companies in the Vox Telecom Group and as DataPro, as
an ISP, does provide a bulk mailing service to customers as well, there
is a possibility that you are on one of our customer's databases.

In order to investigate this properly I would really appreciate if you
could forward me the "February newletter" to which you refer so that I
can investigate this thoroughly for you.

I look forward to your response.

Regards,
Maggie

I have forwarded the most-recent offending email -- "signed" at the bottom by a Mr Gary Sweidan, Datapro's Managing Director, I am sure he is blissfully unaware of the content, or that it is being blasted to a who-knows-how-large list of unwilling , unconfirmed, not-opted-in recipients.

Sadly for them, I redacted out all the recipient email address details and message UUIDS that might server to identify the address it was sent to ;-)

One of the few spammer activities more loathsome than "address repurposing" is listwashing -- removing the whiners from your list whilst blithely continuing to spam the quite ones who Just Hit Delete.

15 February 2008

Taking on the Spammers: Datapro/Vox Telecom - Part 1

For well over a year now I've been getting spammed by Datapro (a Vox Telecom subsidiary) with sundry Friendly Newsletters, Product Offers and Special Crap We're Sure Will Interest You.  Now we're in a fight argument complaint-resolution discussion.

Background

Datapro is a fairly large supplier in SA of web and email hosting, ISP services, and all the myriad little bitty services around that.  They're also one of only 15 "Large" members of the Internet Service Providers' Association -- the industry's self-regulation watchdog in SA -- and hence a signatory to ISPA's Code of Conduct, which includes a clause saying, in effect, "members won't support spam or spamming."

I have never been one of Datapro's customers because I think their technical standards are... dodgy... to say the least.  But, I have had contact with some of their technical staff in the course making changes to email, web-hosting and DNS on behalf of some of my clients who do use Datapro as their service provider.  For whatever misguided reasons.  Evidently, some of Datapro's tech staff have had their email address-books "harvested" by The Marketroid Department.  Or Something.  How ever it happened, my email address got repurposed without my knowledge or prior consent.  A major point, here, is that I have never been in a business relationship with this company.

In the anti-spam world "repurposing" is considered a Very Bad Thing, and will result in instant and permanent blacklisting on some aggressively well-run mail servers.

I've lost count of the number of times I have emailed the sender asking, demanding, pleading or threatening legal action, in the interests of getting off their mailing lists.  Countless times I've clicked on the (rarely present) "unsubscribe" links and jumped through web-page hoops to get unsubscribed.  Nary a confirmation have I received.  Nor has any of this actually diminished the volume of crap I get from them.

To add insult to injury, Vox Telecom, the parent company, have in turn taken to spamming their subsidiary's lists.

A Lightbulb Moment

A short while ago, a contact on one of the local Internet-industry mailing lists I haunt, suggested that I lodge a complaint with ISPA.  I must confess that I had never seriously thought about it, but maybe worth a try...

I waited.  Made sure I gathered and archived the evidence.  Then, last Tuesday, I struck: lodged a complaint via the ISPA's webform:

Action The First: The Complaint
NameISP: Datapro/Vox Telecom
name: <redacted>
email: <redacted>
Address: <redacted>
Telephone: <redacted>
Cellphone: <redacted>
SectionCoC: E. Unsolicited bulk mail (spam)
Details:

I have never been a customer of Datapro.  My only interactions with them have been on behalf of my clients, in the course of managing clients' DNS, email, hosting, etc. technical requirements where those services have been provided (at the clients' choice) by Datapro.  As such my interactions have been with technical service personnel only.

During the course of such interactions Datapro staff have, without my consent or prior knowledge, added my email address to various mailing lists that they use to send marketing "newsletters" and advertisements (a.k.a. address repurposing.)

I have on numerous occasions requested that my details be removed from all mailing lists and databases under Datapro's control to no avail.  I have made such requests telphonically, by email, and by clicking through the (rare) unsubscribe links that some of this spam contains.

Finally I have records good enough to prove my point.  Their latest "February newletter", sent in duplicate today, 9 February 2008, is in clear violation of

1) my past instruction to them of 2 August 2007 (and subsequent, evidence-free removal-link-clicking)

2) the ECT Act itself, in failing to meet the information provision and opt-out requirements of the Act, and

3) the ISPA Code of ConductCopies of all relevant emails are available from myself.
Let's see what results...
Related Posts Plugin for WordPress, Blogger...