In On a Small Screen, Just the Salient Stuff they make an interesting observation that web browsing on an iPhone can actually be superior to its big screen cousins:

A quick trip to Web sites like Facebook, Twitter, Zillow or Powerset, all of which have been redesigned to take advantage of the iPhone, makes it clear that bigger is not necessarily better when it comes to exploring cyberspace. By stripping down the Web site interface to the most basic functions, site designers can focus the user�s attention and offer relevant information without distractions.

This sounds great at first. Dump clutter. Get me just what you think I need. Be ruthless. Edit the world for me. The problem is you know what will show up fist are the most popular and profitable items. So in aggregate we end up seeing much less of the world that we would on a big screen . It's the small world phenomena where new nodes are more likely to attach themselves to existing popular node and we'll see that same power law type friend and follower figures we see on Twitter, Facebook, and FriendFeed. A few people have many 1000s of followers. Most have close to none. And the world is far smaller than it would have otherwise been because it is being viewed thrugh a small screen. To control information flows you'll just need to control the first small screen full of information because like for Google search results, few people venture past the first page.

So small iPhone screens mean the rest of us are banished even further out in the long tail galapagos.

A popular image of the internet it to think of it as a vast interconnected intelligent organism, like this gorgeous map of the internet. If the image is true then who is the heart, the brain, the liver, the nervous system, or the alimentary canal? I'm sure you have candidates for each role :-) The immune system though I think is us, we the citizens of the internet. And we as the internet's immune system react to DRM as an invading organism: destroy ! destroy! destroy!

The parallels between the arms race between invaders and the immune system and DRM and netizens are uncanny. The job of an immune system is to detect foreign invaders and destroy! them. This is harder than it looks. Distinguishing between an unwanted visitor and your liver takes some careful work. But once your immune system figures out what's you and not you, it attacks.

Some organisms have figured out a way to beat the system by waiting a while for the immune system to detect them as foreign and then rapidly evolving by changing their DNA. This causes the immune system has to start the whole process all over again. We were all brought up on microevolution. Evolution happens by small point/insertion/deletion changes. But the scientist Barbara McClintock discovered something radical, that genes could jumparound chromosomes. This means genetic changes could be radical and immediate, not small and slow. Of course Ms. McClintock was ostracized by the scientific community for speaking a heresy. But modern science finally caught up to her, many decades later, and she won her Nobel prize.

J Wynia wrote an interesting post It's Time to Decouple Your Development Process where he talks about one of the most useful patterns a group of developers can use, it's a pattern I know as the Golden Spike Pattern. The name comes from the 1869 ceremony that drove the final spike completing the transcontinental railroad in the US. The railroad linked the eastern US with the western US, finally meeting up at Promontory in Utah. Constructing a transcontinental railroad is a complex time consuming business. They made it work by researching a general route for the railroad, acquiring the necessary property, and then having two sides building their parts of the railroad independently,

The metaphor behind the pattern name isn't subtle. The idea is groups don't have to be in lock step dependency to create a complex system. All they need is a general plan and an agreement of where to meet up. In software a meet can be an API, schema, library, protocol, or a test suite. Once that meet up point is established all the different sides can go about there business without interdependencies. From a lean manufacturing point-of-view this is a powerful way of increasing overall system flow because it's dependencies that cause wasted time.

Each side can work out their own issues in their own way, as long as they meet up. And how would you like spend a decade building a railroad and not have it meet up at the right place!

In the past few years we've seen a huge rise of successful systems built following a Document Object Model
(DOM) type of architecture. By that I mean: open systematized models of complex domains that are easy for applications to specialize and extend in a cooperative manner.

This approach has quickly taken over the tradition libary + statically compiled language paradigm in amazing products like Eclipse, Aspect Oriented Programming (AOP), DOM + Javascript + DHTML, and in Content Management Systems (CMSs) like Drupal.

While we must pay homage to Smalltalk for its unified image based development environment, the most familiar example of the DOM architecture runs inside your favorite browser, using the dynamic trio of DOM + Javascript + DHTML.

The olden days of interface writing is represented by the applet. You combine all your libraries into a single application and download it to an interpreter. This is basically the same as compiling to an image and running the resulting executable on your favorite OS.

Look how different is your browser's DOM based approach. The DOM provides a really rich and functional model of a document, your web browser, which is made available to all applications running in your browser. Multiple libraries can be directly loaded into the browser and build on layers of each others functionality. Applications can add new methods, events, and properties directly to the model. And powerful meta tools like jQuery are creatable because of the dynamic power and regularity
the browser environment provides.

In the US family sit-down dinners are a cherished tradition. Many Americans have fond and powerful memories of sitting down around table with their family at night and eating together. I realize other cultures have very different dinner traditions, but I was struck how much like a family dinner is the morning scrum meeting.

In a family dinner there is a sense of community. You are surrounded by people who support you, who are there for you, and who will help you when times turn tough. During the day you battle the world and then you come home to the comfort of the family circle that is the dinner table. You gather together. It's a time to talk to each other, reconnect, and figure out how everyone is doing. Every gathering is a ritual act of building community.

Scrum meetings create a very similar sense of family and community. Without a regular meeting we are just individuals carrying out tasks like machines. It's being part of a team that helps elevate work to life.

Imagine if you will that car rental agencies rented cars like programmers are hired at most software companies...

Agency : So sorry you had to wait in the reception area for an hour. Nobody knew you were coming to today. I finally found 8 people to interview before we can give you a car. If we like you you may have to come in for another round of interviews tomorrow because our manager isn't in today. I didn't have a chance to read your application, so I'll just start with a question. What car do you drive today?
Applicant : I drive a 2002 Subaru.
Agency : That's a shame. We don't have a Subaru to rent you.
Applicant : That's OK. Any car will do.
Agency : No, we can only take on clients who know how to drive the cars we stock. We find it's safer that way. There are so many little differences between cars, we just don't want to take a chance.
Applicant : I have a drivers license. I know how to drive. I've been driving all kinds of cars for 15 years, I am sure I can adapt.
Agency : We appreciate your position, but we can only take exact matches. Otherwise, how could we ever know if you could drive one of our cars?
Applicant : Oookay. I've driven a Taurus before. You probably rent those, don't you?
Agency : Indeed we do. What year did you drive?
Applicant : It was 2005...but I don't see how that ma...
Agency : Oh sorry, we use the 2006 model. We can't possibly let you drive a later model.
Applicant : But, but they aren't that different. Surely if I can drive a 2005 I can drive a 2006.
Agency : Sorry, sir. Our requirements clearly spell out that you must be able to drive a 2006 model.
Applicant : I've driven a 2006 Escort. Do you rent those?
Agency : Ah, excellent, you are in luck. We have one in stock.
Applicant : Great. Can I rent it?
Agency : No, no, no. We have to go through our interviews now. I'll go try and find the first person.

Around 2,500 years ago the Buddha may have created one of the first and longest lasting intentional communities, that when looked at in a certain light, looks an awful lot like a modern agile organization.

In Karen Armstrong's excellent book, Buddha, we learn soon after the Buddha became enlightened a rich merchant gave him some land and built a few huts so the Buddha and his followers could make a permanent camp. This organization was called a Sangha and was the forerunner of the Buddhist monasteries we see today. Until this time the Buddha and his followers were always on the road and took shelter wherever they could when the monsoon season rained in. Few would travel on the monsoon muddied roads so the Sangha became a place where Buddha and his followers could wait out the monsoons and continue their studies. Soon Sanghas were being created everywhere as appreciative students donated land to the Buddha's cause.

Whenever you get people together you need rules of governance. How should people interact? Who should be in charge? What are the duties and obligations of someone who has joined? What are the Buddha's answers to these questions?

I was surprised. The Buddha advocated decentralized administration and individual authority and responsibility. Since Buddhism is based on taking personal responsibility for finding ones own enlightenment, this organization makes sense, but for some reason I thought he would be more of a waterfall guy than an agile guy.

Which interaction in a design meeting do you think will turn out the best results?

Scenario 1:
Alpha Geek A: That is the stupidest idea I've ever seen. Only an idiot could think up that idea.
Alpha Geek B: What you do mean? It worked on my other projects and it's based on proven patterns.
Alpha Geek A: This project is much more demanding. It has to scale infinitely and cost nothing to deploy.
My new O(1) algorithm for distributed miracle working will do that no problem.
Alpha Geek B: The only scales you could find are on you highly squamata like epidermis. My
framework, though it will take 2 years to develop, will enable us to easily change
our design without changing code.

Scenario 2:
Alpha Geek A: That is the stupidest idea I've ever seen. Only an idiot could think up that idea.
Hub : A, your new algorithm is very creative. It looks like we'll be able to scale
easily with a lot less effort than we do now. Testing will be easier too.
But B has some good point about how difficult it will be to make changes
fast in the field. What in particular don't you like about B's idea and
why don't you cover again what particular goals you are trying to
accomplish?
Alpha Geek B : Wait, I see A's point. Let's call up Maven A. I think we can work
out how to get the best of both worlds.

This amusing and disturbing post on C++ Threading paints C++ going the same way Java went, into a corner. Having been on more than a few large distributed programming projects I have kept careful tally on how many people can get large complex threading projects correct. After a long and careful analysis the results are clear: 11 out of 10 people can't handle threads. It gets too complex too fast. I do not have fMRI studies showing brains literally melting when thinking about multi-threaded programs, just many many anecdotes.

I recall fondly one of our most experienced engineers creating a lock that was off by one instruction and how that 1 in a million error took weeks of debugging by a tiger team to find. The release was eventually canceled. I recall how I took a lock on a data structure that when the system scaled up in size lasted 100 milliseconds too long which caused backups in queues throughout the system and deadly cascade of drops and message retries. Very embarrassing, but even more difficult to anticipate. I can recall how having a common memory library was an endless source of priority inversion problems. These same issues came up over and over again. Deadlock. Corruption. Priority inversion. Performance problems. Impossibility of new people to understand how the system worked.

Yet when it all worked it worked so beautifully the pain was almost worth it. Almost. It's like the image of the City on the Hill. Almost within our earthly grasp, but just out of reach. Yet we keep striving, hubris driving us ever forward in her many threaded chariot.

This is the advice of Barry Schwartz, author of the Paradox of Choice. You can see the video of his presentation at google at http://video.google.com/videoplay?docid=6127548813950043200.

One the big points of his book is that "the more choices available the more likely you will choose nothing." This has enormous implications for life as we know it, but as this is nominally a programming blog, I also see it having a lot of relevance to program design.

One of his recommendations to overcome the problem is to use opt-out instead of opt-in. When you ask people to opt-out of becoming organ donors, for example, you get a lot more people becoming donors. When you ask people to opt-out of joining a 401K you get a lot more people joining a 401K. The idea is that when people do nothing they get what's probably in their best interests. More people becoming organ donors and joining 401Ks are good things.

How does this related to programming?

At the application level we usually do a pretty good job of providing default options that give a good user experience. Can you imagine MS Windows, for example, by default providing no security and expecting the user to turn on all the security options? It would be nightmare. Oh, wait, that what they do, isn't it?

Yet some default options perplex me. Why doesn't my editor by default save my changes all the time so my changes are safe from a power outage? I have to dig through 10 layers of menus to turn this work saving feature on. I usually remember to turn it on only after I have lost an hour's worth of work. So maybe at the application level we could do a better job of providing a "good things happen" experience.

Every so often I read someone who thinks because they do unit testing with 100% coverage they don't need to do any other type of testing (regression, system, acceptance). What could possibly go wrong if all the units work?

One way to think of a program is a program is like a chemical reaction. A bunch of particles react to make a product. An interesting feature of chemical reactions is the resulting product has nothing to do with the original particles. I think about this as the role model for emergent properties in general.

Take as an example our old friend H(2)O: water. We bathe in it, we drink it, we throw it on people when they catch on fire. Let's take a closer look at the components of water.

Water consists of hydrogen and oxygen. By looking at the properties of hydrogen and oxygen individually could you predict the properties of water? Let's see.

First take a look at this movie of the Hindenburg exploding. The Hindenburg was filled with hydrogen and as you can see it is highly flammable.

Oxygen itself doesn't burn, but oxygen is required to make everything else burn. Add oxygen to a flame and you get a lot more flame.

So when you combine hydrogen and oxygen what do you get? Shouldn't get a compound that burns and explodes into a fiery hell? You would think so, but you get water instead.

To recap:

          hydrogen      - think Hindenburg
       + oxygen         - burn baby burn
       ----------
       =   water          - not fiery hell

The properties of the elements doesn't tell you what will be the properties of the resulting compound after the reaction.

Time spent creating mock objects is often wasted. The "T" in
Test Driven Design is just as important as the "D."
Real tests--ones that find bugs--require testing real code.
Emphasizing making fast tests using mocks misses what is most
important: creating code that works in your deployed environment.

If your code passed a unit test using mocks but failed
in a system test, you would get no sympathy from me.
Your shit must work, i don't want to hear about the rest.

This may sound stupid, but things are what they are.
If you have a rule like tests must be fast, and apply
that rule blindly, then you are not paying attention
to what things are, you are just paying attention to
the rule. This causes you to justify dropping testing
in favor of test speed.

For the same reason people say you don't need to test
setters and getters, i don't really find a lot of problems
with incorrect communication with other classes. For
all the pros of the design part of TDD, i still want
to find bugs and to find the "real" bugs you need
to test the real code. If that takes time then it
takes time.

Favoring mock creation means i am spending considerable time
creating tests that skip about a gazillion failure interaction modes.
That's time i would rather spend on finding real problems
in real code and creating real code to solve real problems.

You want to find problems in unit tests rather then system or
acceptance tests because bugs are much easier to find, reproduce,
and fix in the unit test environment. If it is being said problems
will be found at higher levels of test then i think you can do away
with most unit testing period because you can always make this
argument.

I do use mocks, have for many many years, but using mocks
is a last resort for me, not the way to do everything.

The above response is in reaction to:

> Seth Ladd:

David Bau has an interesting post on when one should choose Encapsulation or Representation. http://davidbau.com/archives/2003/11/12/encapsulation_or_representation.....

Use encapsulation within a subsystem. Use representation between subsystems.

Encapsulation creates a language API binding. This is the least
flexible option when trying to integrate between subsystems.
CORBA, RMI, RPC, etc have all pretty much failed for system
integration for this reason.

For example, i want to use Perforce via a programmatic interface.
They have a C++ API that runs on platform X. I use perl and i am
on platform Y. Within Perforce they should use a carefully
crafted encapsulation. If they had used SOAP or simple HTTP
at the subsystem boundry i would be set. These are available to me
in every language on every platform.

Encapsulation makes for a great internal program architecture.
Representation between subsystems makes for great accessibility
from any language and platform.

Perforce has a command line program called p4 which people often use
to make wrappers around perforce. The problem is this is neither encapsulation
or representation. The output must be screen scraped because it is meant
for CLI display. This is a far cry from having a well defined schema with
specific fields and values. With perl's regular expressions screen scraping
isn't horrible, but it is still crude and error prone.

Frameworks get a bad wrap because everyone has a story about how they were on a project that tried to build a framework and it spiraled out of control and the whole project failed and everyone died a firey death. I contend frameworks fail for pretty much the same reason any other software project fails. If it's not done properly it will fail. If it's done properly yet get a huge ROI.
From dictionary.com:

frame·work Pronunciation Key (frmwûrk) n. 1. A structure for supporting or enclosing something else, especially a skeletal support used as the basis for something being constructed. 2. An external work platform; a scaffold. 3. A fundamental structure, as for a written work. 4. A set of assumptions, concepts, values, and practices that constitutes a way of viewing reality.

There's no reason a framework must apply accross multiple applications, there's no reason for it to be OO based, and there's no reason for it to be complete.

My definition of a framework in the context of programming would be something like:

The systemization of a domain expressed in code to solve a particular class of problems in a particular ecology.

The framework could be large or small. It could work in one application or many applications. The primary point is a framework allows developers to solve their problem in terms of the framework. If done well it can provide a lot of leverage (ROI). A framework doesn't solve all problems in every application.

They keys are: 1. Systemization is an experienced based process otherwise the probablity of success is reduced greatly. Experience comes from working on the same or similar problem in multiple projects. It never stops which is why a framework is never perfect and is never done. Systemization is a key to success in other fields and it can be a key in software.

Bala Paranj wrote: Hello, If I have an abstract class Bird and subclasses Sparrow, Pigeon and Ostrich having methods fly(), with no-op for the Ostrich fly() implementation. I am violating the LSP. Is this violation acceptable? Is there any case where the violation is acceptable?
My Reply:

It's up to you. LSP is helpful in programming because it let's programmers more confidently reason about programs because they can assume they know how something will behave. LSP is not a constraint in the real world, it is a constraint that can be put on software to help build systems.

One root of the problem is that types in most programming languages are based on classical categorization. The real world is much more interesting and may combine examplar and prototype based categorization (http://www.bsos.umd.edu/hesp/newman/Newman_classes/Newman300/webpages/ca...).

A tree stump can be considered on the outer boundry of being a seat, but it's not our best example of a seat. And in fact the only reason a tree stump could be considered a seat at all is because we are human. In the classification system of an ant or an elephant, a tree stump probably wouldn't be in the seat category.

This sort of ambiguity is not easily represented in class structures.

The categories we think of as natural and intrinsic to the world are largely the production of our human embodied self's relation with the world.

In the same way categories in your system need to be related to the problem space. The problem space is what embodies your solution and acts as the context in which questions, definitions, and relationships are resolved.

If you wish to build a software system then following LSP is probably a good idea. Figure out why you care about flight and make all your objects consistent with that view of the world. Trying to model reality in an unsuitable medium like a programming language can get very frustrating. Perhaps the brain is the complexity needed.