Archive for January 2nd, 2009

[warning: This is a fairly “duh, I know all this” post for some people.  I just wanted to get out an accumulation of thoughts on the topic.  The interesting stuff is closer to the bottom. ]

It’s pretty safe to say that Google is the current king of technology and innovation.  No it doesn’t have everything figured out and it isn’t even as big as HP, IBM and Microsoft, but it does currently have one of the most ubiquitous product on earth, Google.com.

Competitors and up and comers want to catch some of that Google fire or at least it’s riches.  Most of them are trying to beat Google at its core -Web Search.  Some of that is out of fear that Google can encroach on their core business and some of it is out of envy of the profits Google created.  These are real factors.  Real big money is at stake… but…

Google can’t be beat at web search.

But wait!

It doesn’t need to be done!

Beating web search won’t happen with better web search, we don’t need a better way to search web pages or YouTube videos. Google can be beat in business and will be “beat” when the environment is right for the next big shift in info tech.

Consider that no one tech company has managed to be number 1 in more than 1 major aspect of the modern info tech chain.

Server OS is linux/unix

Database is Oracle/MySQL

Enterprise connectivity are the big infrastructure companies

Consumer connectivity are the cable and phone companies

Server hardware is HP, Dell, and Sun

Consumer OS is windows

Browsers are IE and Mozilla

Web server software is apache

Productivity software is Adobe and Microsoft

Web Search is Google

Email is your ISP, business or prefered portal

Social Network is MySpace or Facebook

and so on…

The point is while most everyone chases Google on Web Search (and Search Advertising) they are missing the point that the cost to make a competitive product and maintain competitiveness is just not going to be worth it.  Users will always search the web, but by now it’s just like everything else in the digital chain – it’s one of the functions that will steadily grow in use each year but the massive usage and monetization explosion is over.   You can see this in Google’s financial trends.  Anyone who jumps into the fray at this point is just going to be swept up in the general trend of steady growth. (i.e. the cost of entry is still high, but the upside isn’t high like it was before)

The key to massive wealth generation in technology is to create the next function that users aren’t already using but will need/want as soon as the tools/technology/culture reinforce it.

A classic example is the Operating System.

Microsoft whipped everyone at the consumer OS.  However, Microsoft’s tech dominance didn’t come to an end (some will claim it hasn’t ended) because someone built a better OS (even though Apple did in many eyes). OSes all got to a certain usefulness and now they are practically given away (on netbooks, phones, etc.).  Few people pay full price for an OS any more because it just is part of the package.  The OS isn’t the killer app anymore, no matter who makes it.  Microsoft’s dominance was done in by the accumulation of improved hardware, connectivity, great web apps, reliance on non pc devices, web servers, more savvy user bases, open source… literally, the OS just became another low margin part. (yes, I know they still make billions on it, but really it’s not the big margin it used to be and its getting MORE expensive to remain competitive).

Recent Examples of this concept include Social Networking leapfrogging AOL and chatrooms, Twitter and Facebook overtaking IM, search targeted text ads dominating banner ads, timeshifting and on demand doing in TV guides and one time broadcasting…

Google has shown that it can no longer build the Next Thing.  It can buy or extend the Next Thing once others have built, but raw creation is not going to happen again at Google.  Its web search (and related Ad Sense) business takes most of its energy and contributes 97% of its revenue.  Maps, Earth, Picassa, YouTube, etc. etc. are neat, but they weren’t invented or dramatically improved by Google.  Sure, Google’s mass in web search made these products runaway hits, but none of these products make Google profits.  It’s 20% rule for engineers is legendary, but that 20% has generated products that mostly a revisions on what others have done.

This is true of Yahoo and Microsoft and Apple and HP.  And it’s all good.  They are incredibly profitable businesses that will continue to generate profits and make strides in their products.  They just will never again generate the massive insta-wealth that they did when their core products made a splash in the market.  The fact that they are all now established businesses with product, marketing and shareholder obligations keeps a good amount of creative energy tied up in just maintaining the core business.  Start ups don’t have these pressures and so they usually make the killer apps.

What’s the next big thing?

One thing that’s becoming clear is that we have so much data (even our data generates data) that just finding data is not going to be enough.  We needed web search as soon as the number of web accessible resources outgrew the directory approach.  We wanted better social functions when chat rooms and IMs couldn’t give us the disired threaded discussions and access to media.  We now have access to millions of scanned books, every piece of video media created in the last 10 years, everything our friends are doing through out the day, all our IM conversations, all the emails we send, every SKU, all academic papers, code, genetic data… etc. etc.  Finding is not the issue.  Any of us can FIND relevant data.  None of the innovations after Google search have been much more than ways to digitize data and provide a way to find it. (Facebook, YouTube, GPS, Wikipedia and so on are just specific implementations of the basic digitize and make findable)

The next problem to solve is DOING something with this data.  The technology has to start DOING stuff for us.  Not just presenting summaries of options for data or links to more data or a list of more media.

What should I watch? Not, just what can I watch.

What should I buy? Not just what can I buy.

Who should I vote for? Not just who’s on the ballot.

What does life expectancy and cost of health care trends imply and what should I do about it? Not, here is are the latest numbers.

Some might call this artificial intelligence, others call it smart computing, others call it computation.  Call it whatever you want.  The technology needs to start doing things, not just sorting and filtering.  Analyzing, deciding, contacting, buying, reserving and so on.  In small ways it does do that.  Music playlist systems, spam filters, virus protection, news alerts, tivo, algorithmic stock trading all compute on the data and take action.  These are terribly complicated functions nor do they dramatically impact how we live, but they are glimpse of what’s to come.

When you think about advertising (which is what pays for Google’s technology), that’s what it’s about getting you to make decisions and take action.  Both the user need and advertiser need come together at the point in which data is sythesized and acted upon.  Although a commercial flop, Facebook’s beacon was a daring attempt to bring advertising closer to this vision. Unfortunately for Facebook the user need wasn’t quite there and the implementation didn’t really DO anything other than create more data for users to look at.

To bring it all together – I hope internet innovators stop trying to help us index and find more data.  Google and the other providers do that well enough.   Personally, I don’t want to spend my time searching for info.  I’d rather be creating or doing, not just sifting, browsing, surfing, filtering.  And I want to be creating and doing interesting things, not mundane things like scheduling, routing through city streets, paying bills and so on.

P.S.  Slightly off topic but relevant if you want to think through why Google can and will be overtaken by another tech company as king of the hill… it’s built on arbitrage.  30% of it’s revenue comes from Ad Sense ads it runs on other sites.  However, most of those other sites generating that 30% get 50-80% of their traffic from searches on Google leading to their pages.  Facebook, MySapce and Wikipedia are the rare exceptions that don’t need Google’s traffic and don’t generate big revenue (Facebook and Wikipedia don’t carry Google Ads at all and it pisses Google off).  This is unsustainable.  The marketing is just stuck for now ituntil advertisers demand more than tiny text links on mostly bad web pages.  For now, that’s the best way to advertise online.  It won’t stay that way.  The banner ad gave way to the search ad.  There will be something else.  Even if it doesn’t replace the search ad, it will chew into its profitability.

Worse though is that the majority of the google ads are purchased by professional arbitragers.  SEM firms, ad agencies, traffic specialists.  They know how to buy clicks on google for .20 and charge advertisers/clients 1.00.  As the tracking tools get better and more clients increase their knowledge, this scheme will breakdown.  The prices for advertising are going to come way down making the arbitrage game very difficult.  Without professional arbitragers playing on google, Google will lose a lot of revenue.

You can uncover this for yourself.  Trace when Google’s revenues shot through the roof.  It will coincide with them releasing ad features like “keyword replacement” which allowed big advertisers to insert the search keyword into generic copy.  This meant that millions of search ads went online that really weren’t all that useful, but they had the right search keyword in the copy.  You can see this at play today when you search for bizarre keywords and you’ll find and Amazon.com or Ebay ad.  Click on it and see if there’s a real page there.   Or consider an advertising like the one in this article.  The worst possible scenerio – a professional arbitrage outfit that does reverse mortgages.  They do $100,000 a month in ad words with Google.  Do we really think this is going to last?

You can also correlate the google revenue growth with the advent of SEO firms who put up tons of worthless pages and helped legit publishers put up tons of worthless pages.

Lastly, because of how important Google’s traffic is to many online businesses, the general functionality of the web is less than it could be because everyone tries to make their pages Google search friendly.  Google has actually significantly stunted the growth of useful web functionality.  However, that trend is reversing with runaway, non Google supported successes like the iPhone, Facebook, Wikipedia, widgets, twitter and wordpress.  Google is very slowly losing it’s grip as the only way to get traffic.  And with experiences like Facebook and Twitter we’re seeing better functionality.

I’m not holier than thou.  I’ve made plenty of money playing with Google traffic.  I use Google non stop personally.  It’s not going to go away.  It’s not going to one day be running at a loss.  It’s an essential company providing much needed and wanted services.  My point in this post and the PS is that the margins and valuation they have isn’t sustainable at all.  2009 will showcase some of cracks as the ecosystem they’ve helped create begins to morph under the drastically different world we live in.  This isn’t like after dotcom crash.  This is bigger and Google’s one revenue stream is not market proof.  It benefited from Google.com being the only game in town for finding stuff and Ad Sense being the only reliable source of paid advertising online.   Advertisers aren’t going to pay the same amount for ads AND Google’s dominance on web traffic isn’t going to remain.

Read Full Post »

[UPDATE  1/10/09:   This post is fairly popular so I assume many people are looking for help on Scramble Squares and potentially hints or solutions.

Sadly, there is no way to shortcut these puzzles.  I suppose if you look at the box, and your box shows you the completed puzzle, you can “cheat” that way.

If you just want to use an already coded “solver”, there are a few out there.  Here’s a good one.

Read the instructions on how to encode your puzzle pieces so the solver can work properly.

Note that puzzles can sometimes have more than 1 solution.

Also note that the algorithmic solvers are not “shortcutting” the puzzle.  They simply try more options than you do much faster.  It’s pretty much just brute force.  A modern computer can run through all 23 billion combinations in 30 seconds (and usually does it faster because it might pick the right center piece early on… which is what the popular algorithm uses as a starting point).

That said, humans can somehow solve these puzzles without “trying” a lot of configurations.  As I discuss below I have some theories how this is possible, but I do not have a definitive answer.]

Here are  some papers, code and blog posts (this is nice one too) on how to algorithmically search for solutions to squzzles/scramble square puzzles.  You might have received one of these puzzles as a holiday gift over the last few years, they were quite popular.

C and Perl Implemenations

General Backtracking Approach

The algorithm is straightforward – just one that searches through solutions.

What’s interesting is that I’ve seen people solve these puzzles, even brand new ones (no prior knowledge), very quickly.  There’s something that happens with a persons vision or something that’s helping them not have to exhaustively search the full solution space.  If I’d seen someone do this once or twice, I’d think it was just lucky picks. (these puzzles have enormous solution spaces (4^8 x 9! = 23,781,703,680 puzzle configurations) )

Is there something in this puzzle that “hints” to a human early in the solution testing that a solution is viable or not.  That is, after 1 or 2 pieces placed, can the human see a promising solution “faster” than the basic algorithm that searches quickly through all piece placements and orientations.  If so, what is that data (“hint”) the human sees and how can we factor it into the algorithm?

Possible hint data:

Rules of thumb on how all these puzzles are printed and cut (do the puzzles all get made with same orientations so exposure to one puzzle provides data on other puzzles?)

Humans can see the whole pattern in parallel even when pieces aren’t lined up so they don’t have to check each piece systematically

Are combinations of pieces eliminated as the humans solve it thus taking them out of future solution attempts, reducing solution space the more the human works on the puzzle

Read Full Post »

Well, isn’t that nice.

Unconscious plagiarism is valued as wrong by people who only see black and white. ‘Wrong’ in the sense that someone is taking credit for something that was generated or created by another person and the interloper was not giving credit for the source of the new idea or ‘thing.’

Do you know who wrote the “Dick and Jane” book series or what the best way is to grill lobster tail? How about the basics of the scientific method or the definitions of reinforcement or punishment? Right…you don’t remember. Yet you have them and you didn’t create them so you must have got them from someone and you need to give that person credit.

However some versions of plagiarism when we write may be due to inattention rather than to intention. When people who use the words or thoughts that are not their own they call it plagiarism. Those questioned about copying or using another’s words and ideas without giving them credit invariably state that their behavior was ‘unconscious.’ They were unaware of their copying when they used those words or ideas as they used. When unintentional plagiarism happens it is referred to as cryptomnesia.

This “concealed recollection,” is the name for a theoretical phenomenon involving suppressed or ‘forgotten’ memories. It refers to cases where (apparently) a person believes that he or she is creating or inventing something new, such as a story, poem, artwork, or joke, but is actually recalling a similar or identical work which he or she has previously encountered. The term was addressed extensively by Federal Judge Richard A. Posner in his book, ‘The Little Book of Plagiarism’.

Could there be any clearer case for learning?

As humans we learn at an intense rate not even closely appreciated by most academicians or those who spew content on the different media channels. We do it for all of our lives. And all the learning we do is not equal. In the thunderstorm of what is learned from before we are born and throughout intense learning periods and even during less intense periods there are millions of discriminative stimuli [SDs] that get linked with the environment surrounding the paradigms of antecedents and consequences. These morph and are reshaped repeatedly. We mix them together regularly as in a “mixed metaphor” such as “It is pitch quiet in here!”

The SDs that come to control the different elements of what is leaned become diluted differentially over time due to conflicting cues [SDs] as well as disuse of the information.

In the case of cryptomnesia the person is supposed to bookmark or otherwise categorize and account for the reading in Dr. Suez that led to thinking that there was a button maker that used the same color combinations. Or, was the person watching TV’s American Idol supposed to categorize statements [time, date, person, context] made by the stars on the show so that when a viewer’s book was written 3 years later, “Skill follows Will” credit could be given for the title of the book written about athletics?

I hope you consider the quagmire that this level of accounting requires. It may generate a set of people that refuse to write, compose, paint or speak if the logic is carried comes any more pervasive than it is.

Blogs appear to be somewhat beside themselves when it comes to references and original content. Yet the American Psychological Association has made a science out of referencing and citing what one close friend has called, “old dead men” to the point where we reinforce citing less ideas for what of having to explain where we got our more profound ideas.

Sounds like learning is the same as cryptomnesia in that it represents the case of integrating what is consumed in books, lectures, friends, movies, neighbors, parishioners, colleagues, TV or anything else. It can be ascribed to someone other than its original creator (another interesting myth) or another condition and be treated as intuitive or common sense…

Consider the following:

In a conversation with colleagues [over beer, martini’s, scotch, margarita’s or spring water] at [the office, racquet club, saloon, grocery store, etc.] you outline an idea that is not in the mainstream of the company. You go over the usual dichotomy of good vs. risks and why or what is the net of the idea.

It is considered and absorbed by the others and when appropriate, it is tied up and put into a package in some way that allows it to be identified. Comments like

“…well it is something to think about…”

“…I think it was tried in 2004 and never got a sponsor…”

“…there is good reason that it won’t work… don’t bring it up again.”


“…let me have a synopsis of it and I’ll bring up at the manager’s meeting in June.”

When providing an idea or approach, strategy or process to others they often absorb it and subsequently either

  • deny its validity
  • provide its validity but question its significance
  • after time has past bring it up as if they just thought of it.

Who should get credit for the idea?

is the originator in the loop in the above example?

Do we really want to live like this when data and information grows faster than our global national debt?  Isn’t there a better way?

Read Full Post »

In a previous article I suggested that it becomes incumbent on the reader, listener, watcher or any engaged person to be able to tell when something in the media didn’t seem right or justifiable, etc. as in CNN and Evil – Snivil… I promised those that there are some rules of thumb for detecting faulty, deceptive or malicious content. The one selected is the Sagan Baloney Detection Kit. There are dozens of them out there on the web, in science methodology texts and even some in writing books. Like any set of rules of thumb, they are not absolute but provide an approximation that will save time and angst when sifting through the escalating volumes of content we have access to.

Like every source of ‘help,’ use what works for you and toss the rest. Know that those that want your eyeballs understand this list better than most of us and do what they can to keep you from recognizing these red flags in their materials.

Let us know what we missed and what we need to cull from our list.

Baloney Detection Kit: Based on selections are taken and similar to those in a book by Carl Sagan “The Demon Haunted World” Ballantine Books (February 25, 1997) ISBN-10: 0345409469

These collectively or individually are ‘red flags’ that suggest deception. The following are tools for detecting fallacious or fraudulent arguments wherever they present themselves.

  1. Wherever possible there must be independent confirmation of the facts
  2. Encourage substantive debate on the evidence by knowledgeable proponents of all points of view.
  3. Arguments from authority carry little weight (in science there are no “authorities”).
  4. Try not to get overly attached to a hypothesis just because it’s yours, your parents, etc.
  5. Quantify, wherever possible.
  6. If there is a chain of argument every link in the chain must work.
  7. “Occam’s razor” – if there are two hypotheses that explain the data equally well choose the simpler
  8. Ask whether the hypothesis can, at least in principle, be falsified (shown to be false by some unambiguous test). In other words, it is testable? Can others duplicate the experiment and get the same result?
  9. Conduct control experiments – especially “double blind” experiments where the person taking measurements is not aware of the test and control subjects.
  10. Check for confounding factors – separate variables impacting the conclusions.

Common fallacies of logic and rhetoric

  1. Ad hominem – attacking the arguer rather than the argument.
  2. Argument from “authority”
  3. Monocausality: Cause and effect statements
  4. Argument from adverse consequences (focus on the dire consequences of an “unfavorable” decision; attack a sovereignty or you’ll be fighting them on the streets of New York).
  5. Appeal to ignorance (absence of evidence is not evidence of absence).
  6. Special pleading (typically referring to god’s will, Buddha’s mysteries or passions of Islam).
  7. Begging the question (assuming an answer in the way the question is phrased).
  8. Observational selection (counting the hits and forgetting the misses as in fortune telling).
  9. Statistics of small numbers (such as drawing conclusions from inadequate sample sizes).
  10. Misunderstanding the nature of statistics (President Eisenhower expressing astonishment and alarm on discovering that fully half of all Americans have below average intelligence!)
  11. Inconsistency (e.g. military expenditures based on worst case scenarios but scientific projections on environmental dangers ignored because they are not “substantiated”).
  12. Non sequitur – “it does not follow” – the logic falls down.
  13. Post hoc, ergo propter hoc – “it happened after so it was caused by” – confusion of cause and effect.
  14. Meaningless question (“what happens when an irresistible force meets an immovable object?).
  15. Excluded middle – considering only the two extremes in a range of possibilities (making the “other side” look worse than it really is).
  16. Confusion of correlation and causation.
  17. Straw man – caricaturing (stereotyping, marginalizing) a position to make it easier to attack.
  18. Suppressed evidence or half-truths.
  19. Weasel words – for example, use of euphemisms for war such as “police action” to get around limitations on Presidential powers. “An important art of politicians is to find new names for institutions which under old names have become odious to the public”

Read Full Post »

Planet Google by Randall Stross

This book is two things: boring and generic.  As with so many “tell all” business books, this is nothing more than a collection of press release and business articles glued together with a not-so-insider narrative.

Stross provides us little insight and only leaves us with the rather benign self prescribed prediction from Google that it will take them 300 years to organize the worlds info.

For anyone who’s paying attention to business on the web, this book, at best, serves as a summary of Google’s well known product releases, legal bungles and proclamations.   There’s no unique analysis of Google’s approach to intellectual property, no suggestion of better web crawling methods, no digging up of data center floor plans nor any interviews with users, government officials nor ex employees – any of which would have given us info would couldn’t otherwise dig up ourselves.
Google is by far one of the most difficult business set ups we’ve seen in history.  Most internet users and knowledge workers depend on Google to make their living yet we all know the dangers of one company controlling so much access and housing so much data.  This is a topic worth 50 books and yet we get only snippets of discussion on that in this book.

Hell, Stross doesn’t even question how it’s going to be possible for Google to last the 300 years when it only makes money by selling basic text ads.  Are we all to seriously believe text ads can provide revenue to power this company for the next 15 years of technology changes, legal battles and employee costs? Much less 300 years?

This book could have easily attacked Google deeply on its complete bullying of intellectual property holders from libraries to the TV networks to Wikipedia authors.  Google, for all its Don’t Be Evil, continues to thrive on the idea that if you get big enough (push enough traffic) you can do whatever you want with IP.   They, perhaps, didn’t set out with that as the driving force, but it is what powers their business.  Perhaps you might not even agree with this assertion.  However, indirectly they have created a nasty underbelly market on the internet where lots of publishers, site operators, content “creators” all operate under that notion.  Consider how much of its revenue comes from spammy, stolen content pages with Google ads on them, and you’ll see my point.

Stross doesn’t touch on any of this.

He seems to be so enamored with the idea of a company organizing the worlds info and is invites to company meetings that he doesn’t consider any of the deep implications of what it actually means to organize info, what it means to us to have a commercial operation organizing it, and how one actually can live by a motto of “Don’t Be Evil.”

I read this book in 5 hours.  Don’t bother reading it.  Pull up your browser, hit the business publication sites and Search Engine watch and you’ll get better insight.

Hey, and maybe for perspective use Yahoo!, Live.com, Ask.com to search.  You’ll have done more homework than Stross.

Read Full Post »

Descartes’ Bones by Russell Shorto  is a lively, curious tracing of the antecedents to the current arguments of the interaction of faith and reason. Shorto concludes that extreme views on faith and reason miss the mark in their own world views – somewhere in between the extremes we can find answers. The thesis from Shorto draws forth from the historical account of what happened to Descartes’ remains. This is a very clever approach both as a story telling device and as a way to contextualize so many of the anchor-less statements we have today regarding faith and reason.

The book reads quickly as Shorto is a capable writer. The historical record is sufficiently deep and he avoids digging too deep of a philosophic or rhetorical hole that plagues so many other popular science history books. The basic tracing of the ownership of Descartes’ remains is incredibly bizarre and even without story telling embellishment makes you want to read to find out what happened. What makes this book stand out is the curiosity it inspires to go find out more about all the tangent plots and philosophic journeys. For example, we learn brief details on the creation of museums. I’d never really thought about when and why museums – strange places when you think about it – came about in the course of humanity going about its business.

I don’t at all agree with Shorto’s conclusions on the implication of the never ending battle between faith based worldviews and scientific reasoned worldviews. I do agree that the search for absolute truth continues to invite bizarre behavior – like the trading and use of Descartes’ skull as a relic and to make “scientific arguments” about brain size and intelligence. Shorto did inspire me to drag out a copy of The Method and remind myself about why a book written 350 years ago continues to be used a reference for modern thought and science. And though Descartes didn’t invent the mind-body philosophy he did package it up nicely enough to make sure some people now sign their emails with “I think, therefor I am” In fact, Descartes did such a good job of marketing the mind-body view of man a good portion of practicing scientists and most of America still believes in the Mind and that the body is a machine.

Read this book.

Read other historical records on Descartes, modernity, the French Revolution and early mathematics – you’ll be surprised, perhaps horrified, at how much of 300 year old thought still shapes our culture.

Read Full Post »