Wikis and Knowlets - A Concept Web for Knowledge Representation

Stumbled upon this site today. Seems to be taking the wiki based collaboration one level higher.

Wikiprofessional’s Concept Web Initiative is a global collaboration to innovate how knowledge is represented and expanded on the Internet.

Knowlets

The Knowlet summarizes the relations between the concepts and presents the strength of the relationship based on a value derived from three main factors: factual (F) statements found in scientific databases, the co-occurrence (C) of two concepts in a text, and a predictive (P) parameter based on the conceptual overlap of the two concepts.

An extension to MediaWiki

The Wiki is an extension of the MediaWiki software that enables Wiki editing capability in a relational database structure.

Just the beginning

The first incarnation of the wiki is for Life Sciences called WikiProteins.

While the Life Sciences are the starting point for the Concept Web, the Wikiprofessional collaboration intends to expand the knowledge representation and expansion dynamics of the Concept Web systematically in disciplines and languages throughout the world.

This will be an interesting development to watch. With an impressive list of Collaboration and Research Partners, WikiProfessional seems to be off to a good start.

Many questions remain.

• How will this be different from DbPedia and other similar efforts?
• What is the connection to the Semantic Web?
• Is the code an open source like MediaWiki (WikiProfessional is based on MediaWiki)
• Is Knowlets the right meme for knowledge representation? In all fields of interest?

Golden Rule of Documenting Software Design

In the Next Programming Skill You Should Learn, Scott Hackett talks about the ability to communicate well, especially in writing. I totally agree. The post is certainly worth a read.

An unexpected bonus in this post is Scott’s tips on the golden rule of documenting software design.

The golden rule of documenting software design

Try to document whatever you are working on. It doesn’t have to be the worlds most perfect UML and you don’t need an expensive tool to do it. In fact, Wordpad and Paint are sufficient, and don’t tell me you don’t have those. Write in a way that best expresses your intent and your thought process in coming to the design decisions you did. I have a golden rule of documenting software design:

Describe your design to others as you would have others describe their design to you.

When you spend the time to do this, you’ll find that many benefits will follow. You’ll get feedback from others that may have tried to solve a similar problem and have insight you may not have thought of. You’ll leave a clear trail to follow for those that work on the code after you. Most importantly, you’ll shed light on your work, which everyone who depends on your work will appreciate. Even if no one else reads what you write, you’ll still have worked through problems in your head that can only lead to better design in the long run. There is no downside to documenting your designs.

One of the experiments I am conducting with a set of interns is to make them write a LearnLog and ProjectLog - just a few bullets or short sentences. We do this in a wiki so that every one in the project has access to it.  It is also one of the easiest and most effective ways to communicate asynchronously with the team.

In the Project Log, they are supposed to write the following:

• Design decisions
• Decision to use libraries/tools and why
• Problem areas and stumbling blocks and how they solved it
• A list of links to the resources they used

It is still an uphill battle but we are getting better gradually.

Wikis and Information Intelligence

Information Intelligence is the practice of gathering intelligence useful to an organization. It uses Open Source Intelligence to enrich an organization’s ability to gather intelligence for internal use.

Open Source Intelligence (OSINT) is an information processing discipline that involves finding, selecting, and acquiring information from publicly available sources and analyzing it to produce actionable intelligence. In the Intelligence Community (IC), the term “open” refers to overt, publicly available sources (as opposed to covert or classified sources); it is not related to open-source software. OSINT includes a wide variety of information and sources:

Media - newspapers, magazines, radio, television, and computer-based information.

Public data - government reports, official data such as budgets and demographics, hearings, legislative debates, press conferences, speeches, marine and aeronautical safety warnings, environmental impact statements, contract awards.

Observation and reporting - Amateur airplane spotters, radio monitors and satellite observers among many others have provided significant information not otherwise available. The availability of worldwide satellite photography, often of high resolution, on the Web (e.g., Google Earth) has expanded open source capabilities into areas formerly available only to major intelligence services.

Professional and academic - conferences, symposia, professional associations, academic papers, and subject matter experts.^[1]

In addition to these Media mentioned above there are several sources for Web Data Mining. There are several aspects of improving Information Intelligence:

1. Gathering information from a variety of openly available sources
2. Supplementing the open source intelligence with internal information
3. Providing a collaborative platform to share information
4. Enriching information - tagging, interlinking, annotating
5. Versioning information to keep it current
6. Providing a semantic layer for easy retrieval and integration with other tools
7. Providing both a horizontal view and specific vertical views of the information

Wiki is an ideal tool for managing Information Intelligence inside an organization. You can start with a base wiki technology like MediaWiki (used by Wikipedia) and build additional layers like Semantic Media Wiki or provide structured data access like DbPedia . You can get information on several vertical sharing information sites using MediaWiki here.

A good example of both horizontal and vertical views is demonstrated by the US Government initiatives Diplopedia and Intellipedia.

Recent congressional testimony from Jimmy Wales, the founder of Wikipedia,^[5] notes the difference between vertical and horizontal information sharing and suggests that both could be successful e-government endeavors. Intellipedia is an excellent example of sharing information horizontally across agencies, and Diplopedia has found similar success in sharing information within the Department of State bureaucracy. Statements on both wikis encourage cross posting of relevant information as appropriate.

Wikis provide a great foundation for Information Intelligence. Enriching Wikis with semantic annotations, providing more powerful viewing options, granular addressing and increasing the quality of links may go a long way in increasing their effectiveness.

Meta:

This entry was triggered by an email invite to an Intellipedia session at the MIT Center for Collective Intelligence.

Wiki Trends

 This is an article I wrote for I.T. Magazine. It appears in their Nov 07 issue. They were kind enough to let me put this on my blog.

Wiki Trends

You have probably heard about wikis; some of you have probably used Wikipedia or other wikis. In this article, we assume that you know a bit about wikis. We do not cover the basics, or how to create your own. Instead, we will look at some usage patterns, innovations and emerging trends. We will start with a brief history of wikis; look at the way they are used by individuals, organizations and social communities. We will finally look at some of the innovations in both the technology behind wikis, and their applications. We hope that you can use some of these ideas to explore wikis for your own use, whether it is personal, professional, or for your community.

A brief history

The term wiki is a short form for WikiWiki, a Hawaiian term for quick. The original goal of the wiki was to quickly create a set of web pages without writing HTML(Hypertext Markup Language). All you needed was a simple text editor, a few conventions for <formatting. You could just type the text like you do in a notepad. This will give you a minimal web page instantly on saving. That was almost twenty years ago. Wikis have come a long way since then. The first wiki was created as a web-based collaboration tool by Ward Cunningham. He made the sources available free. This encouraged a lot of people to innovate and create various implementations in their favorite languages.

Wiki has some very simple basic concepts:

• A Page is the basic unit of wiki content. You can just type text to create a page. You can easily create bulleted lists, numbered lists, decorations (bold, italics), and tables, by using simple markup.
• WikiWords are automatic links. You can either type camel-case words (a set of capitalized words squished together without spaces, for e.g., CollaborationTool) or enclose them in special braces ([ and ] in some wikis, or [[ and ]] in others). The wiki engine expands wiki words, for example CollabTool into “Collaboration Tool” and looks for a match. If the page exists, it automatically links to it. If not, it marks the wiki word with a “?” next to it when the page is saved. This ? is a hyperlink to create a new page.
• Links and back-links are one of the most powerful features of a wiki. The wiki engine knows about links in a page. It not only keeps track of forward links, but also back links (the page that links to the current page). You can use this feature to find out all pages that link to a particular page.
• Recently Changed Pages is a list of pages that have been recently changed. When you are using wikis for collaboratively editing content, you can go to this list and find out when each page was changed, and by whom.
• A keyword-based search facility allows you to quickly locate information on a wiki. This encourages rapid use as well as authoring. I can go to a wiki, and try to find a topic using the search capability. If it does not exist, I can create it and link it to related pages very easily. The ability for anyone to easily contribute and add their bit of knowledge makes the wiki an easy-to-use collaboration tool.
• Automatic Versioning is another feature that allows people to experiment. Whenever a page is edited, it is automatically saved. This encourages people to add comments, make corrections and add content to a page without worrying. It is easy to go back and edit the page or roll back to the original content. This is probably one of the most heavily-used features in Wikipedia.
• Change Tracking is another capability available in some wikis. If you like a topic, you can subscribe to the page with the topic or a set of related pages. Whenever these pages change, you will automatically receive an email notification.

What makes Wiki a great tool for collaboration?

First of all, most of the wikis are free. This allows people to experiment with them and explore various applications. It is easy to set up and manage. Here are some more reasons why people like to use wikis:

• Easy to set up and operate
• Easy creation or editing of pages
• Easy to upload documents and images and format content.
• Easy to link pages
• Easy to interconnect with other wikis
• Easy editing by users. This allows multiple people to collaborate and create documents.
• Built-in search facility
• Available on a wide variety of platforms
• Some wiki engines like Mediawiki are highly scalable (several million pages edited by hundreds of thousands of authors)
• Since most of the wikis are open source, they are easy to customize.

A Wiki mindmap

The diagram below provides a brief visual overview of a wiki. We focus on two aspects – how Wikis are used (also known as wiki applications) and innovations in wiki space.

There are broadly three types of applications for which Wikis are used – Personal, Organizational, and Social. We will discuss each one of these in more detail.

Personal Wikis

Personal wikis can be either desktop products or browser plug-ins. The desktop products are like any Windows or Unix applications. They are used by a single person on a desktop. Browser plug-ins allow you to use the wiki as a browser side-bar, or one of the popular deskbars.

I use personal wikis to keep notes, to keep track of web-based resources, as an idea log, and a journal. It beats a notepad any time. People also use it as an authoring tool, a documentation aid, and even for writing books. There are tools that let you take random pages in your wiki, organize them into a cohesive sequence of pages, and output them in popular formats like Microsoft Word, DocBook, PDF or HTML.

Why are personal wikis better than notepads? First, all your notes are in one place. The pages can be easily interlinked (using wiki words or wiki links). But the most useful feature is the ability to search through a large volume of text and quickly locate the pages we need.

Organizational use of Wikis

Increasingly, more organizations are using wikis for sharing information. For example, the HR (human resources) group can use a wiki to keep employee manuals, easily update them incrementally, and make them available on the organizational intranet.
Since many wiki engines (the software that powers a wiki) allow you to have multiple instances of wikis and cross-link information, each department/group/project may have their own wiki. A small group sharing information, documents, and knowledge in their own space can be very productive. As a software team, we use wikis to manage specifications, share design ideas, brainstorm implementation options, and even create product documentation. In many organizations, there are internal knowledge bases available only to the employees and partners, and external knowledge repositories available to customers. The last time I looked, every major open source project had their documentation in Wiki format. Even Microsoft Developer’s Network (MSDN) has a wiki. The same is true for eBay, Cisco and Google Code.

Wikis are human editable. There are tools that let you store discussion threads as pages, send email to create pages, or store blogs in wikis (a blog-wiki combo is sometimes known as bliki).

Social Wikis

The best example of a Social Wiki is Wikipedia. It is a global encyclopedia maintained by hundreds of thousands of people worldwide in multiple languages on a voluntary basis. In fact, Wikipedia’s information about people, places, and events is so rich that now there is a new project called dbpedia that takes the information in Wikipedia and makes it available as a database.
Social/Public wikis have a lot of promise; we have not even scratched the surface. Can you believe that MediaWiki, which powers Wikipedia, is developed and maintained by a team of just seven people?

Wiki innovations

Wikis are over twenty years old (the first one was started in March 1995). While the adoption of wikis took a while, they are rapidly increasing in popularity. So are innovative variants; we will cover a few of them here.

Swicki
A swicki is a collaborative search engine. It uses one of the most powerful features of a wiki – the wisdom of a group, also known as collective intelligence. This is an innovative use of the collaborative aspect of wikis to help search better. You can create your own swicki. From http://www.eurekster.com/swickibuilder/faqs.htm:
Sometimes, looking for specific information has that needle in a haystack feeling. Not only can you get 6,000+ results from a simple query, the most relevant data for you can be buried way down the list. Swickis lets you slice and dice and customise your search engine query so that you can specify the most relevant sources, then get further refinement of the results once like-minded users start engaging with the results. Every click refines the swicki’s search strings, creating a responsive, dynamic result that is both customised and highly relevant.

WikiHow

My favorite social wiki is WikiHow. It is a universal repository of “How Tos”. Do you want to know the chat lingo? There is a page that provides links to the information. Do you want to build a web site? Just go to WikiHow page (http://www.wikihow.com) and type “web site” in the search engine. A page pops up with useful links to information. Are you an expert in making paper airplanes? Go ahead and create a “how to” page. WikiHow uses Wiki’s collaborative authoring to provide you a very innovative application.
WikiDocs and WikiTrails
Some wikis now provide a feature called WikiTrails. WikiTrail is a page that simply contains a list of other pages. WikiTrails allow you to create a virtual book out of a large body of information in a wiki without physically reorganizing it.
A set of authors distributed world wide jointly write Wikibooks by working on different topics and work as both writers and editors. These wikibooks are freely available to be shared with others.

DBPedia
Wikipedia contains a lot of useful information. But it is growing at such a rapid rate, it is difficult to keep up with it. So there are tools being developed to extract structured information. DBpedia is one such tool. It is a community effort to extract structured information from Wikipedia and to make this information available as a set of RDF (Resource Description Framework) triples. Several Semantic Web tools like SPARQL can then be used to perform complex queries on this data.

The DBpedia dataset currently provides information about more than 1.95 million “things”, including at least 80,000 persons, 70,000 places, 35,000 music albums, 12,000 films. Altogether, the DBpedia dataset consists of 103 million pieces of information (RDF triples).

Wiki Widgets
As wikis get more sophisticated, the quality of the wiki content gets more diverse. Swicki already allows multi-media content on its wikis. Other wikis support the concept of widgets – little objects like calendars, video clips, audio into a wiki page.
IBM has taken this one step further in their QEDWiki. They provide a way to drag and drop widgets on a wiki page and create Mashups.

Wikiness
I don’t think this is a real term. I coined it to indicate a certain behavior of an application that has wiki-like features. As wikis become more popular, and people feel comfortable using them, you will see wiki-like features in other products. Wikia and Swicki already do that.
Resources:

• Wikimatrix : A cool resource for comparing different wikis
• DbPedia: An indication of where wikis are going in future
• wikibooks: Collaborative authoring of books using wikis
• Swicki: Collaborative Search using Wiki
• AskWiki: When wikis become big, this is what happens

Little Innovations: Plone 3.0 and Wikiness

I have been talking about some of the wiki attributes for normal text. For lack of a better term I called it Wikiness. It is nice to see that as one of the new features added to Plone 3.0 which was released recently. From the Plone.org:

Lets you use the wiki markup to create and link documents in any content.

Plone takes a novel approach to the wiki format — incorporating support for it in all existing documents. That means that we support wiki markup in any type of content — be it plain text, any of the text-based markups, HTML — it even works with Word and PDF documents if you have display transforms set up for these.

A common complaint about wikis is that they have weak access control, and promote bad site structure. This new approach to wiki in Plone means that you can get the benefits of the quick and easy wiki content generation and linking, housed in a world-class content management system with proper access controls and the multitude of features exclusive to Plone.

It’s wiki, without the wiki aftertaste!

It is one of the many little innovations in this new version of Plone.  Other notable innovations include support for Microformats. Here is a list of new features in Plone 3.

Wiki’s Group Mind

I was browsing a Python mailing list - a discussion about best programs completely written in Python. Not sure how I wandered off that topic. I think I went into list of wiki engines and landed in this page. Then I read Martin Pools:

I think in some way it is the closest I have come to WabiSabi in code

I did not know what WabiSabi was. So I clicked on the link. And then another and then another. Here are a couple of steps of this journey. You can make your own.

Wiki is an eternal now.

It is timeless. Anything can be deleted at any time, but everything that has not been on RecentChanges for a while stands on equal footing. Nobody has done anything like this before. Wiki’s growth will outstrip any attempt to change it. Wiki is its own history. It is a good history, and it will guide growth over time because it is a living document of what Wiki is and has been.

In some respects, Wiki’s group mind works just like a human’s:

• RecentChanges represents Wiki’s conscious mind and short term memory - what it is thinking about right now.
• Wiki’s subconscious and long term memory is represented by all the past pages that are related to current topics, and connected to them by links/associations on Wiki as well as in the contributor’s/reader’s heads.

Sometimes, Wiki seems a little unconcentrated (several seemingly unrelated, “off-topic” pages on RecentChanges). But be careful not to judge prematurely - maybe it’s just Wiki’s subconscious mind at work, and these things are subtly related. If Wiki can’t make the connections, the thought will quickly vanish. — FalkBruegmann

There is something beautiful about reading this - a feeling difficult to describe. There is a wealth of information in this wiki. It is true wiki’s group mind at work. Explore it. Try Abstraction, for example.

Cool Tools for the Web - Part-3

Some times I use a tool for a while before it appears in this list.

StumbleUpon

I stumbled upon, StumbleUpon a while ago, while tracking the visitors to my blog. It is one of those compelling applications that you have to have. It is sitting in my Firefox Toolbar, right now. StumbleUpon made news this week. They were acquired by eBay. They seemed to have added more services too. Definitely worth checking out.

Swicki
Swicki is a cross between wikis and search engine (not sure why they decided to spell wiki as wicki). The swickibuilder lets you build a search (a few terms and information sources) and use it collaboratively. According to the swicki FAQ:

A swicki is new kind of search engine that allows anyone to create deep, focused searches on topics you care about. Unlike other search engines, you and your community have total control over the results and it uses the wisdom of crowds to improve search results. This search engine, or swicki, can be published on your site. Your swicki presents search results that you’re interested in, pulls in new relevant information as it is indexed, and organizes everything for you in a neat little customizable widget you can put on your web site or blog, complete with its very own buzz cloud that constantly updates to show you what are hot search terms in your community.

You can check it out here. I built a couple of them to play around with. There is not that much activity in my swickis. You may understand why, if you look at my swickis and the popular ones

Enterprise 2.0 and Rich Internet Applications

What’s SAP doing to help customers infuse Web-based apps with rich-client functionality?

It’s nearly impossible to achieve a RIA architecture - even on one of the many RIA stacks - unless you’ve done a decent job of service-orienting the underlying capabilities you want to access. You need the metadata that describes not just the business process execution, but also the metadata that describes conceptually what the service is doing, how it can be used, and how it should adapt to different device capabilities in order to give you a rich application. That could be on a PDA or a smart phone as well as on widgets, Flash-type experiences as well as dynamic HTML Ajax-types of environments.

According to SAP Enterprise 2.0 has these multiple aspects that need to work cohesively.

• Rich Internet Applications (RIA)
• services-oriented architecture (SOA),
• metadata,
• model-driven architecture (MDA)
• event-driven architecture (EDA)

Front end technologies (Web 2.0 functionality):

• portal,
• widgets,
• RSS,
• wikis,
• blogs and
• Ajax technologies

Another two interesting trends mentioned in this Q&A are that:

• The run times are in Flash
• The development environment is NetWeaver

Look at the list of TLAs (three letter acronyms). You can read more in this Q&A: SAP on Rich Apps, Architecture and AJAX based Portal Environments.

Web 2.0 and Semantic Web

From Tim O’Reilly’s My “Outdated View” of the Semantic Web:

It’s always seemed to me that Web 2.0 as it was evolving would eventually turn into the Semantic Web, just that it was too early to specify the means by which it would do so. What’s at issue is not where we’re going, but what tools we will use to get there.

This is a great post of Tim quoting the comments he received about his notion of Semantic Web from some of the semweb luminaries. I think it is easy to come to the conclusion that Semantic Web is complex and rigid. Hopefully many of the emerging use cases would convince people otherwise. Widespread adoption of Semantic Media Wiki may be a good starting point.

Robot Companions

This article, got me thinking about how we can use robots. First a couple of stories.

My dad had a stroke a few years ago and his short term memory is shot. In a half an hour conversation, sometimes he asks the same question about 5-6 times. I visit him occasionally and talk to him, but do not spend as much time with him, as my mom does. I have to really praise my mom’s patience in dealing with this condition of his. But even she gets tired some times. My dad probably realizes this at some level. So he does not talk much. A companion robot would really help my dad and others like him. He can converse with it and since it will never be tired of answering the same question again, he can probably spend more time chatting. It may even help his condition.

I have seen inquisitive kids asking lots of questions. They ask everyone - parents, grandparents, teachers, uncles, and any one, they can get an answer from. After some time, the kids stop asking questions. They realize that asking questions is not really enjoyable, since they may not even get the right answers. A pet robot that can talk and answer questions would be a great asset to the family. Children won’t be limited by the lack of time or inability of parents to answer questions. I wonder whether a pet robot would increase children’s knowledge since the answer to every question may lead to lot more questions. But more important, children may grow up with the feeling that it is OK to ask questions.

There are lots of poor villages in India, China, Thailand, Indonesia, Africa,  and several other places in the world. Many kids in these villages don’t go to school. They have to work in the fields or do other jobs to make a living or support the family. A cab driver, I recently talked to, told me that he stopped studying after 3rd grade, because he had to earn a living. There are probably millions of people like this in the world. What happens if each village had a few companion robots. Kids and elders can walk up to them and ask questions, and get answers.

I was having a conversation with my 13 year old nephew. He is not very happy with the way his classes are conducted. And asking questions is not encouraged, since the teachers have to stay on schedule, to cover lessons. He is a bright chap and does well in his studies. So I believe him, when he says that he is very disappointed with his level of learning at school.

I think a robot can help in all these situations. It can be a great companion, if it can listen and ask a few questions and carry on a semi intelligent conversation. I am quite sure it can be programmed to never tire of answering questions. My dad, the inquisitive kid, the cab driver, the village boy and my nephew can all use this robot. The robot need not even be sophisticated. If it can understand speech, and answer questions, that will be more than sufficient. The programming of these robots is not easy. But I am sure if we start now, over the next 5-10 years, it will get a lot better. It can also use something like the Wikipedia to field questions and give answers. Of course we may need a WikiSchool which is oriented more towards teaching content, but there are already several efforts to provide learning content in wikis.

What are the essential characteristics of this robot?

• Can look like a human or ET or some non threatening species (may even have pet versions for kids)
• Should listen, ask questions and give answers
• Remember people (by voice or look) so that it can remember their learning style and the learning history
• Should be able to look up some kind of structured knowledge using Wikipedia or other similar resources
• May optionally have a screen that can show some animations or movie clips to make a point
• May be based on something like AIML (artificial intelligence markup language)

These robots should not cost much and the cost can go down as the volumes go up. Can the world use these robots? You bet. Will they threaten any one? Not likely.

Update:

I just found this posting on Digg about the Top Ten Robots You Can Buy. Some of them definitely have the potential to be companion robots even though the price points are quite high (like Nuvo).