LinkLog: Software

This blog is such a wonderful source of information. Here are just a few of the posts:

How a beautiful idea becomes a Frankenstein system is a must read for every software developer. Here is a small fragment of a much more comprehensive diagram in this post.

I always thought that we built tools to take boring repetitive parts of the work and automate it. How Tools Frame Programmer’s Mindset makes you reflect a lot more about the tools. What qualities should effective programmer tools have? The author identifies three:

• Usability - enhance flow of programmer’s ideas or at least don’t impede and interrupt this flow.
• Representation - enable easy for understanding and modification representation of the structure, ideas and domain concepts in the code.
• Agile development friendly
  

From Beginners to master programmers - First Language and More is a problem that faces every training organization. When I started working on Learning Point, this was one of my constant worries. I have seen several threads of discussions on the choice of first language for programming.

This blog post is a good starting point. Hopefully I will have more to contribute after my current experiment with 5 interns for the next six months.

1. Train clear logical thinking.
2. Understand modern software concepts and environments.
3. Learn to effectively implement customer needs.

Only Speak When It Improves the Silence

From  Flash Coder’s Etiquette:

Each email is a request for the time of other people… they have to spend at least some time with it, even if it’s just to delete your message unread. If you post too much, without offering corresponding value for the reading, then you risk diminishing the community, and risk getting yourself killfiled in the process.

That may be a good test for blogs, as well. I wonder how many of my posts qualify.

TED Talks: Beauty Matters

Murray Gell-Mann talks about Beauty, Symmetry and Emergence  in his TED Talk: Beauty and Truth in Physics (video). It was one of those high quality TED presentations you get the privilege to watch once in a while.

• Beauty Matters
• Math Matters
• Symmetry Matters

Murray also concludes the 20 minute talk with a description of “Emergence”:

YOu don’t need something more to get something more. That is what emergence means.

Life can emerge from Physics and Chemistry and lots of accidents.

The human mind can emerge from neurobiology and lots of accidents.

The amazing thing about this talk is that beauty can be a test for a good theory. Listening to this talks certainly gives you a lot to reflect about. It suddenly dawned on me that a lot of what he was saying, applies to software, as well.

Solve the meta-problem

I keep getting links to cool articles, blogs and essays. Sometimes I just mark them to read later. Some times I just dive in, sample them a bit. What excites me, may not excite you. But if you are in the software industry and share some of my interests, you may just want to take a look.

My sources of essays are links on Digg, doggdotus, slashdot, reddit. There is a bit of overlap there (doggdotus is an aggregator of digg, slashdot and del.icio.us). Here is an essay that got me started for the day.

How To: Be More Productive

Aaron is accomplished. Long before I read anything he wrote, I knew him for his software contributions. He describes himself as an Activist, Writer and Hacker. I knew about reddit and web.py and his contributions to semantic web and Python communities. I recently started reading his Raw Thought

This blog post is a gem. Like many other posts of his, it makes you reflect. If you are a software developer/software entrepreneur you can relate to this essay . For me, there is one thing that stands out more than anything else:

Another way to make things more fun is to solve the meta-problem. Instead of building a web application, try building a web application framework with this as the example app. Not only will the task be more enjoyable, but the result will probably be more useful.

Yeah. That is what it is all about. If you are a software developer, you can be an order of magnitude more productive, if you can take the approach of solving a meta problem. Some times this method of solving produces a tool. Some times it produces a design pattern or a framework. But almost, always, it produces one of the most elegant, reusable solutions.

Here are a few meta problems and the solutions that software pioneers invented. Some times they are so beautiful, you are sit in amazement at the mind that created them.

• Parse different formats - Build a way to describe a parser and build a parser-generator (lex/yacc)
• Find a way to describe a database - Recursive use of the concept of relationship to describe themselves. The metadata in an RDB are just tables like any other (with some special privileges)
• You need a way to build different markups - Build a makup meta language (like XML)
• You need to describe a set of resources and relationships and make statements about them - Create RDF/RDFS

Need I say more?

Virtual Spaces as Third Places

The concept of third places is a new one to me. Thanks to Konard for blogging about Classrooms as Third Places and providing some useful links.

Third Places Oldenburg suggests that beer gardens, main streets, pubs, cafes, coffeehouses, post offices, and other third places are the heart of a community’s social vitality and the foundation of a functioning democracy. They promote social equality by leveling the status of guests, provide a setting for grassroots politics, create habits of public association, and offer psychological support to individuals and communities.

Oldenburg identifies that in modern suburban societies time is primarily spent in isolated first (home) and second (work) places. In contrast, third places offer a neutral public space for a community to connect and establish bonds. Third places “host the regular, voluntary, informal, and happily anticipated gatherings of individuals beyond the realms of home and work.”

While both Konrad and Oldenburg identify physical spaces as third places, I wonder whether a virtual spaces (blogosphere, facebook like social communities) can be considered third places too? They definitely “host the regular, voluntary, informal, and happily anticipated gatherings”.

Evolving Wikis

When I Googled Wiki, I got more than 412 million hits. While Wikis have been around for more than 10 years, their visibility has increased only in the last few years - mostly thanks to Wikipedia.

So where do wikis go from their initial role as collaborative tools? Let us look at both some current and many possible future developments.

1. Wiki Scripts
Integration of scripting into wiki lets you build wiki applications. Twiki is a great example of this. Open source products like Moin-moin and commercial products like JotSpot do this too. Client-side scripting with Javascript and server side scripting may both be supported.

2. Wiki Plug-in Modules
Plug-in modules allow new functionality to be added to the wiki through a simple component architecture. Moin-moin, Jotspot and others have a scheme for writing plug-in modules that let you extend/customize these wikis

3. Wiki Skins
Changing the skin of a wiki allows you to change the look and feel of a wiki. The initial set of wikis were bland affairs. Now there is a lot more color and style being added through custom skins. Just look at something like Wetpaint to get an idea about the trends in this direction.

4. Wiki Templates
The initial wiki had only one type of page. A title and a bunch of text with links. Nowadays most of the wikis have templates you can choose when creating a new wiki page. Here is the description about templates from Moin-moin FAQ:

if you want certain types of pages to have a similar format (similar headings, organization, etc.), you just define a page that ends in Template, and when creating pages of this type, select that template and edit it. The wiki fills in the starting content for you. Templates are editable wiki pages like any other.
To create a Template page, just create a new page called <something>Template

5. Semantic Wikis
This is the one, I am most excited about. Semantic wikis associate model based metadata with wiki pages. This increases the ability for applications to gather knowledge from wikis. Here is what Wikipedia says about the purpose of Semantic Wikis.

As Wikis often serve as CMS or knowledge management tools, Semantic Wikis try to enhance them and allow users to make their internal knowledge more explicit and more formal, so that e. g. it can be searched in better ways than just with keywords.

Some systems are aimed at Personal knowledge management, some more at knowledge management for communities. The amount of formalisation varies: existing systems range from primarily content oriented (like Semantic MediaWiki) over content oriented with strong formal background (like IkeWiki) to systems where the formal knowledge is the primary interest (like Platypus Wiki). Also, Semantic Wiki systems differ in the level of ontology support they offer. While most systems store their data as RDF, some even support various levels of ontology reasoning.

6. Attaching Documents to Wiki Pages
Several wikis allow you to attach documents to a wiki page. Some of them include adaptors that understand different document formats, index documents and allow search. This allows a small business to use wikis as low end content management systems. You can also use this capability to use Wiki as a Personal Knowledge Management tool. I had written earlier about using Wikis as backup for my Outlook email client.

7. Wiki Application Programming Interfaces
Application programming interfaces (aka APIs) allow wikis to integrate easily with other applications. This allows other applications to send data to the wikis and get data from wikis. This will also allow wiki information embedded into other, non-wiki applications.

Here is a Wiki Matrix that cover features of various wiki products. This cool resource, allows you to pick a few wikis and dynamically contruct and view a matrix of supported features.

So where do we go from here? I see a lot of exciting possibilities. Some of them may already be there:

• Wikis as Application Platforms (saw a couple of presentations on this at MashupCamp2 with IBM leading a very ambitious effort).
• Wikis and Microformats - Embedding microformats in wikis will increase the capability for wikis to become a canvas for building semantic webs.
• Semantic Wikis are an exciting development. They are at a nascent stage and you will probably see more activity in this space.
• Wikis and mashups - Imagine a mashup that takes data from a wiki and mashes it with other webservices like maps. A virtual tour guide may let you use a map as the UI, and explore historic events/places combing information from Wikipedia and Google Maps.
• Wikis as Mashups - Wiki as a mashup canvas. This is an exciting idea with a lot of potential. There is similarity with another similar concept - Wiki as an Application Platform.
• WebServices interface to wikis can explode the possibilities of using wiki in ways that we have not thought about. This allows wiki integration into both mashups and enterprise applications.
• Enterprise wikis - This subject deserves a full blog of its own. I can see heavy use in HR, HelpDesk, Organizational Knowledge bases as a start. But enterprise wikis can be used whereever informal collaboration is required - product development, product rollout, marketing campaigns etc. Better than the current method of using emails and less formal than using portals.
• Wikis and XML - The next generation of word processors all support XML as the native format. We can imagine this being the case with wikis too. The benefits of using XML as the underlying data format for wikis are too numerous to elaborate here. A few I can think of include structured content, seamless integration with the next generation document editors (aka wordprocessors), vertical wikis (wikis based on vertical XML vocabularies)
• Wikis and RSS - RSS may act as both input and output format for wikis. Every wiki topic is a potential RSS feed. You may be able to send information into the wiki through an RSS feed too
• Wiki and Search - Most of the wikis currently have a great search capability, but like Google, they are just based on keywords.  The possibilities for search integration are endless. You can search for data as well as meta data. Some of the ideas like page ranking take a very different meaning when applied to wikis.

I probably just scratched the surface of possibilities. I have not even touched upon multi-media wikis, wikis as game platforms, wiki-movies, wiki-previews and editing in multi-media documents and many others. I will expand on some of these ideas in future posts.

Collaborator

You can track my slow progress through The Ten Faces of Innovation by my posts. A couple of  description of Collaborator, I like, in this book.

The Collaborator is the rare person who truly values the team over the individual. In the interest of getting things done, the Collaborator coaxes people out of their work silos to form multidisciplinary teams. In doing so, the person in this role dissolves traditional boundaries within organizations and creates opportunities for team members to assume new roles. More of a coach than a boss, the Collaborator instills their team with the confidence and skills needed to complete the shared journey.

Collaborators know that the race is won in the baton process.

Both these descriptions point to a rare quality (compared to individual brilliance). As I read these chapters, several thoughts pop-up in my mind:

• How can you train people to be collaborators?
• How can you recognize and reward them?
• Is this one of the essential qualities of an entrepreneur?
• What does it take to be a collaborator?

I think collaborators are people who are very secure, and know their inner strengths. They probably do not need the instant gratification or recognition of individual brillanceEvery chapter in this book contain several insights that make you think and rethink your roles at work.

Relational Python

In 1985, we wrote a small relational engine using Lattice C 2.0 on MS-DOS. It was a small team (about 4 of us) in Bangalore, India. We had no background on designing an SQL engine and the only source available to us was the "Introduction to Relational Database Systems" by C.J.Date. We took two decisions. One is to layer the database on top of an ISAM engine and another was to implement SQL as a set of relational algebraic operations (restrict, project, join). We later licensed the technology to The Santa Cruz Operations and sold it for a while as Integra SQL.

When I saw this article by James Tauber, on Relational Python, all those memories came flooding back to me.

The goal is not to try to implement a SQL database in pure Python.
Rather the goal is to extend Python's rich data structures like
dictionaries and sets with additional concepts from relational theory.

I think enriching Python with a set of relational operators is a great idea. There are several interesting possibilities with this approach. If these opeators are recursive and can work fast with in-memory data, they may help faster processing of XPath. This may result in more flexible architectures for buiding XSLT and XQuery and later SPARQL.

I have been thinking about relational Python like capability for a
project I am currently working on. This project requires you to build
an in-memory streaming database and provide continuous queries.
Currently we are playing with a prototype built on top of Telegraph CQ
and are looking at other alternative implementations. Using an approach
similar to Relational Python is definitely one possibility.

Why and What If?

One of the most remarkable and fascinating articles on Why and What If

Why do we need to have employees?

Why do we need people to sell the product, if it is so good?

Why do we measure performance by profit and growth?

What if we measured success by the happiness of our partners and our customers, and by the innovativeness, sustainability and resilience of the enterprise rather than its profitability?

What if we designed the enterprise so it didn’t need to constantly grow to be healthy, and so that it used only renewable energy and produced no waste, no pollution, and 100% recyclable products?

Bordering some times on philosophy, this article really makes you think.

Creation

Creation Hymn from Rigveda

A time is envisioned when the world was not, only a watery chaos (the dark, “indistinguishable sea”) and a warm cosmic breath, which could give an impetus of life. Notice how thought gives rise to desire (when something is thought of it can then be desired) and desire links non-being to being (we desire what is not but then try to bring it about that it is). Yet the whole process is shrouded in mystery.

Where do the gods fit in this creation scheme?

The non-existent was not; the existent was not at that time. The atmosphere was not nor the heavens which are beyond. What was concealed? Where? In whose protection? Was it water? An unfathomable abyss?

There was neither death nor immortality then. There was not distinction of day or night. That alone breathed windless by its own power. Other than that there was not anything else.

Darkness was hidden by darkness in the beginning. All this was an indistinguishable sea. That which becomes, that which was enveloped by the void, that alone was born through the power of heat.

Upon that desire arose in the beginning. This was the first discharge of thought. Sages discovered this link of the existent to the nonexistent, having searched in the heart with wisdom.

Their line [of vision] was extended across; what was below, what was above? There were impregnators, there were powers: inherent power below, impulses above.

Who knows truly? Who here will declare whence it arose, whence this creation? The gods are subsequent to the creation of this. Who, then, knows whence it has come into being?

Whence this creation has come into being; whether it was made or not; he in the highest heaven is its surveyor. Surely he knows, or perhaps he knows not.

Thanks to Life Hacker for pointing me to Ancient History Source books