Have you watched an expert at work? You can sense  a level of comfort and fluency. I often watch expert programmers code, debug, fix problems in software.  I get the same sense watching painting shows on TV. A stroke here, a stroke there and suddenly you have a beautiful creation taking shape, right in front of you.

I always wondered what makes experts experts. I thought it was intelligence or just a lot of experience. It seems to be a lot more than that. I never dug deeper into the subject and suddenly I stumbled into this, while exploring a completely unrelated subject.

People who have developed expertise in particular areas are, by definition, able to think effectively about problems in those areas. Understanding expertise is important because it provides insights into the nature of thinking and problem solving. Research shows that it is not simply general abilities, such as memory or intelligence, nor the use of general strategies that differentiate experts from novices. Instead, experts have acquired extensive knowledge that affects what they notice and how they organize, represent, and interpret information in their environment. This, in turn, affects their abilities to remember, reason, and solve problems.

… the study of expertise shows what the results of successful learning look like…

We consider several key principles of experts’ knowledge and their potential implications for learning and instruction:

1. Experts notice features and meaningful patterns of information that are not noticed by novices.2. Experts have acquired a great deal of content knowledge that is organized in ways that reflect a deep understanding of their subject matter.

3. Experts’ knowledge cannot be reduced to sets of isolated facts or propositions but, instead, reflects contexts of applicability: that is, the knowledge is “conditionalized” on a set of circumstances.

4. Experts are able to flexibly retrieve important aspects of their knowledge with little attentional effort.

5. Though experts know their disciplines thoroughly, this does not guarantee that they are able to teach others.

6. Experts have varying levels of flexibility in their approach to new situations.

Now we know. Or at least, have some theories to explore. The fragments above, were taken from the second chapter - How Experts Differ from Novices of a really fascinating book - How People Learn.

There is an element of beauty versus duty in learning most things. When the task is all duty, you may do it, but you may never like it. Indeed, you may come to hate it and stop altogether when the external forces that keep you on task (your teammates, your sense of belonging) disappear. When you enjoy the beauty of what you are doing, everything else changes.

From Eugene’s Math and Computing as an Art.

I think I finally found my morning reading. I stumbled upon this blog following threads of a controversy about a new CS curriculum and ending up in A Small Curricular Tempest. I spent a few hours, reading many of his posts, before I realized that I spent a few hours. This is what used to happen when I was a teenager and in early twenties. Endless hours of reading, engrossed and not even noticing the passage of time.

Thanks Eugene for making my day a bit better and giving me lots of stuff to read.

Stumbled upon this site today. Seems to be taking the wiki based collaboration one level higher.

Wikiprofessional’s Concept Web Initiative is a global collaboration to innovate how knowledge is represented and expanded on the Internet.

Knowlets

The Knowlet summarizes the relations between the concepts and presents the strength of the relationship based on a value derived from three main factors: factual (F) statements found in scientific databases, the co-occurrence (C) of two concepts in a text, and a predictive (P) parameter based on the conceptual overlap of the two concepts.

An extension to MediaWiki

The Wiki is an extension of the MediaWiki software that enables Wiki editing capability in a relational database structure.

Just the beginning

The first incarnation of the wiki is for Life Sciences called WikiProteins.

While the Life Sciences are the starting point for the Concept Web, the Wikiprofessional collaboration intends to expand the knowledge representation and expansion dynamics of the Concept Web systematically in disciplines and languages throughout the world.

This will be an interesting development to watch. With an impressive list of Collaboration and Research Partners, WikiProfessional seems to be off to a good start.

Many questions remain.

• How will this be different from DbPedia and other similar efforts?
• What is the connection to the Semantic Web?
• Is the code an open source like MediaWiki (WikiProfessional is based on MediaWiki)
• Is Knowlets the right meme for knowledge representation? In all fields of interest?

On a winter day in Boston, I sat through a two hour lecture on B-Trees. There was snow outside and we all sat spell bound as Greg Basset, our instructor taught us how Digital’s RMS-11K (the record management system) worked. The concept of incremental loading, fill factors, splitting data and index buckets and compression of duplicate keys, is still fresh in my memory. I gave that talk several times later in my life when I was teaching the subject. In each repetition, I learned a little bit more, answering questions. A few years later, I had the chance to use many of the ideas when we built C-Trieve and Objectrieve, two record management systems on top of which we built an SQL relational database. That more than a couple of decades ago.

Looking at the Programmers Competency Matrix brought back lots of those memories. I think I am going to make a custom version of this matrix, make a poster and put it up on the wall of my class. A lot of these topics are not needed for the application programming today. But for a few of those intensely curious students, this is a kind of road map.

Meta:

Reddit is a great source of interesting posts, especially for programmers. I should thank the kind soul who posted this link.

I was watching this video of Malcolm Gladwell’s talk over the weekend and enjoyed every minute of it.  Malcolm has the uncanny ability to look at problems in new ways. He was talking about the challenges in finding the right people to hire in various professions. He calls it the “mismatch problem”   According to Malcolm, the problem exists because:

1. Our desire for certainty in a world where uncertainty is the norm
2. The growing complexity of every profession increases mismatch

He points out, how, in various fields from sports to teaching, metrics used to select candidates do not really reflect the reality of the changing requirements of the job.

In software we do the same. We try to measure some of the basic skills like knowledge of software development including programming, conceptual understanding, ability to solve simple problems. The true attributes, however, are more complex and are not easy to measure in a test or in a few interviews. These include the ability to work in a team, ability to learn and communicate, a healthy curiosity and a certain amount of pride in work. Our challenge is to figure out how to train our employees in these  new skills.

This was nice to read. Do you know of any other fragments of cool job ads?

Forrester hires people who are inspired and inspiring. Our people are our product and the source of our intellectual capital, so we welcome and reward intelligence, curiosity, and unconventional thinking.

I am going to add these to the list of The New Skills.

Is it really the End of Theory, as Chris Anderson predicts?

Sixty years ago, digital computers made information readable. Twenty years ago, the Internet made it reachable. Ten years ago, the first search engine crawlers made it a single database. Now Google and like-minded companies are sifting through the most measured age in history, treating this massive corpus as a laboratory of the human condition. They are the children of the Petabyte Age.

At the petabyte scale, information is not a matter of simple three- and four-dimensional taxonomy and order but of dimensionally agnostic statistics.

I do believe, as they say in  AstronomyCast that knowing  “not just what we know but how we know what we know” gives you the ability to reason about hypotheses, models and theories. A new kind of analysis with petabytes of data may lead us down the path. But the examples are far fewer and not very convincing - yet.

I would love to watch this conversation and see where it takes us.

Updated: 1st Jul 08

I did not have to wait long. Here is a great article on Why the cloud cannot obscure the scientific method.

What are the skills required for handling future jobs?

I found this report  from National Academies Press. A few skills listed in this document include:

1. Creative Problem Solving
2. Complex Communication Skills
3. Adaptability
4. Self-management
5. Self-development
6. Systems thinking

From my experience of working across two continents, I would like to add a few more:

1. Critical Thinking
2. Cross cultural training
3. Curiosity
4. Inspired and Inspiring (got this from a job ad at Forrester’s research)

I want to dwelve a bit on “Self Management”. I use it as a primary metric to hire people since I do not believe in supervising or managing people. I found that many people who are great at managing themselves do it because they believe in a cause, passionate about what they do or extremely proud of their work.

How do you develop these competencies? Or even explain why they are important? That is the challenge that requires a lot of innovation in teaching or coaching.

Also See:

Malcolm Gladwell on the challenge of hiring in the modern world.

Meta:

This post was triggered by an email from ACM news about a study done in Ireland. Looking at the study, I felt that some of these problems have global implications. So I started searching on “future skills” India, USA.

From Why Internet is an Innovation - our Best Resource for Going Green by Courtney Webster

As an entirely online resource, the Internet can be used to share information across the world, without ever having to waste natural resources. Online bank statements reduce corporate mailings, while websites like www.photobucket.com allow us to share our pictures without making eco-costly prints. When shopping online, using credit cards and online order confirmation, we save the paper and ink that would other wise be used to print cash and receipts, and without making the trip to the store, we’re reducing our carbon footprint in the process.

There are several businesses that can do this, especially ones with mostly knowledge workers. Courtney points out Project Nvokh, a fascinating effort in going green.

I attended a BarCamp at Chennai last week. I forgot to turn off alerts to the barcamp pbwik and keep getting notifications of new barcamps  I got one on (Super)HeroCamp and it piqued my curiosity. It is one of the more innovative BarCamps I have seen in a while. Here is an excerpt titled “Smaller Ideas” from the page:

The smaller ideas
1. What would it look like if we brought the structure of BarCamp into schools? What about a social network for students, teachers and parents where parents could make sure they could keep up with their kids’ curriculum, teachers could understand a little more about a student’s home situation and students could communicate more on the level that they understand? HeroCampSchoolNet
2. What if we provide local communities with an easy (Ning?) way to create a cross-generational learning network. Schools could tap into locals with expertise, and locals could tap the youth and creativity of students. The retired biologist down the street could help students create and maintain a sustainable organic garden, and students could learn about globalization and teach local businesses how to survive. HeroCampCommunityNet

I am always amazed at the innovative ways tools are used. The concept of BarCamp itself is an innvoative idea. Hosting camp announcements on a public wiki is another. The way the concept of BarCamp is being used builds upon both these. Hats off to people who innovate at every level. This is definitely one of my sources of inspirtation.

I gave a talk on Technology Trends and Gleaning Opportunities at TiE Chennai today. It was gratifying to hang out with the participants and swap stories. I just uploaded a copy of the presentation (in PDF format). Here is the link -  technology-trends-jun2008

I also uploaded a copy on slide-share. Here is the link to the presentation.

I would love to hear from you. I am going to keep updating this presentation and incorporate suggestions. I am also planning to spend some time expand my list as well as tools for tracking trends.

Here is a small sample of trends in software.  This is work in progress. I will keep updating it frequently. Instead of waiting till I have my full list, I thought I may just publish this crude list and get some feed back. Some trends are current (like Web 2.0) some of them are future (Semantic Web). Over the next few weeks, I will revisit and keep adding to the list. If you think some thing should be included here, please add your comment. If you have a blog or discussion on trends, you can add that link too. Some of these trends are great blog topics too.

Each trend is an opportunity (or several opportunities). These trends create new jobs, transform existing jobs and the way we live.

Links:

WebTrends Map 2008 - A clickable Map

I watch my blog stats once in a while (used to be a daily routine once). I notice that there are lots of readers for some of my old posts even though some of them may be outdated. When I noticed this, I stopped covering news on my blogs. I even wrote a post titled Does Currency of Information Matter?

Once in a while, I feel compelled to revisit a post and make some minor adjustments - add some new links or add a comment to updated information. I have been thinking about keeping my posts (at least a few of the popular ones) updated. Here are some ideas.

1. Keep updating the posts with current information when relevant, but keep a copy of the old version. Obviously we need to use the same permalink (so that people see the latest version), but does that violate the concept of a perma link?
2. Keep the old blog but change the beginning to link to a new version of the blog
3. Write a new blog and give a new version number with a link to all the old versions (in case people want to follow entries and comments)

I may try all these different techniques. I am sure that some one already solved this problem. If you did, can you share it here? If not, any thoughts?

Last week, I had the opportunity to give a talk at NECTEC (National Electronics and Computer Technology Center) during my one week visit to Thailand. When I visit other countries, I always find an opportunity to talk to a tech crowd. This one was fun. I was tracing some of the short/medium/long term trends in technology and their impact on learning, teaching, collaboration and how we work. I will cover some of those topics in future blogs.

This was a great group of about 20 people. Most of them (I got a few biz cards) seemed to be Ph.Ds and some of them got their post graduate degrees from USA. A few of them were working in Computational Linguistics. We were brainstorming some ideas on natural language parsing issues of non-English languages like Thai.

I think, at a certain level, all software people are alike. They seem to have similar interests and seem to get excited by very similar things. It may be fun to explore Social Networking for Software Developers, internationally.

The presentation itself was just a placeholder. It was just a means of starting a dialog. I was more interested in talking about the application and impact of these trends than the trends themselves.

I went to an eLearning meet a few years ago in Cupertino. I never met any one in the group before and there were some interesting discussion on learning tools. Towards the end I asked them whether any of them blog. The strong reaction, I got, surprised me. “Blogs are for people who do not have anything else to do” said one person. “Who wants to watch pictures of cats and dogs and read people’s rantings” said another. I was not sure what to expect, but these pre-conceived notions gave me all the signals I wanted. I never went back to their monthly meetings.

Why am I recounting this story now? I was reminded of it when I started reading Why Journalists should use Twitter a couple of days ago.

I recently mentioned to a colleague of mine, who also is a freelance journalist, that I’m researching an article about Twitter. “I hope you really trash this service”, was his answer. “This is nothing else than verbal diarrhoea.”

The early adopters are a fascinating bunch. These are the people who are active on Twitter, sign up for several product betas, try almost every product as time permits, read Technorati/Techmeme/ Reddit/ Digg,/eHub/ Slashdot and countless blogs. They remind  me of the robot in the Short Circuit movie that keeps asking for “Input” and devours vast amounts of information.

These are great people to follow on Twitter, blogs and other forums. If you are start-up, these are your little angels. They will tell you whether your product/serviec sucks, give you great suggestions for improvements and if they like your product will tell everyone who may listen to them.

I still have not figured out what motivates early adopters. Is it because they have a high Curiosity Quotient? Or is it because they have a compulsion to make the world a bit better? Or is it something else? These people are one my sources of inspiration.

A great blog post and a discussion thread on reddit. Some snippets (read the blog for a very insightful discussion):

Software that re-defines a category (Google and Amazon come to mind)

Software that saves businesses (and individuals) money (figuring out the benefits to your customer)

Software that helps business earn more money (making it compelling)

Piggyback off where people are already spending tons of money (choosing your marketplace)

Become easier to choose and you become harder to leave (by building and managing excellence)

shrink a market or disrupt your competitors

Get bold initial customers who will take the risk and are willing to share their experiences.

You don’t have to be the guru of an industry; you can often make a huge difference by bringing a computational perspective to the domain (think how you can apply technology to solve real problems)

Find out what they have to do but hate doing and find a way to simplify or automate it.

This is the kind of blog post that I would book mark and read several times, think about it, find more similar ones. It will also be a nice exercise to keep this list some where and grow it based on actual experiences of successful products. Peter Christensen’s articulates so well some of the things I kind of know but never really reflected a lot about.

I think blogs are the best knowledge sharing network you can think of especially If you are lucky to discover ones like Peter’s.

Thanks to @reddit here is the invitiation (dated May 1, 200

I'm inviting the Python developer community to try out the tool on the
web for code reviews. I've added a few code reviews already, but I'm
hoping that more developers will upload at least one patch for review
and invite a reviewer to try it out.

To try it out, go here:

    http://codereview.appspot.com

This post triggers a whole bunch of thoughts and ideas. Clay Shirky talks about Social Surplus and Cognitive Surplus. Some nuggets:

So how big is that surplus? So if you take Wikipedia as a kind of unit, all of Wikipedia, the whole project–every page, every edit, every talk page, every line of code, in every language that Wikipedia exists in–that represents something like the cumulation of 100 million hours of human thought.

The way you explore complex ecosystems is you just try lots and lots and lots of things, and you hope that everybody who fails fails informatively…

The normal case of social software is still failure; most of these experiments don’t pan out. But the ones that do are quite incredible…

People like to consume, but they also like to produce, and they like to share.

It’s [cognitive surplus] so large that even a small change could have huge ramifications. Let’s say that everything stays 99 percent the same, that people watch 99 percent as much television as they used to, but 1 percent of that is carved out for producing and for sharing. The Internet-connected population watches roughly a trillion hours of TV a year. That’s about five times the size of the annual U.S. consumption. One per cent of that is 10,000 Wikipedia projects per year worth of participation.

I think that’s going to be a big deal. Don’t you?

There are all kinds of interesting and useful projects cropping up all over the web. They promote the architecture of participation. Some of them are physical meets, some of them are shared col laboratories mostly powered by wikis and many of them promote sharing. I have my own experiments in this space - one on teaching and learning and another in setting up an incremental innovation lab. I will report my progress in a few months.

]]> http://dorai.wordpress.com/2008/04/27/linklog-cognitive-surplus/feed/ Dorai http://dorai.wordpress.com/2008/04/19/steven-pinker-human-intelligence/ http://dorai.wordpress.com/2008/04/19/steven-pinker-human-intelligence/#comments Sat, 19 Apr 2008 05:25:23 +0000 dorai http://dorai.wordpress.com/?p=855

In this TED Talk,  Steven Pinker talks about the way we use words, how we learn, and how we relate to others.

Human Intelligence consists of:

• A repertoire of concepts (objects, space, time, causation, intention) useful in social, knowledge intensive species
• A process of metaphorical abstraction: conceptual structure bleached of its content, applied to new abstract domains

What is the most important skill to get a lot of passionate users? From  You can Outspend or Out-teach

The most important skill today is… teaching.

Because what you believe in, you can teach. And teaching is the “killer app” for a newer, more ethical approach to marketing. While in the past, those who out-spent (on ads, and big promotions) would often win, that’s becoming less and less true today for a lot of things–especially the things designed for a younger, more-likely-to-be-online user community.

Kind of a markets-are-classrooms notion. Those who teach stand the best chance of getting people to become passionate. And those with the most passionate users don’t need an ad campaign when they’ve got user evangelists doing what evangelists do… talking about their passion.