The Role of News Applications in the New Journalism

Since announcing my intention to accept a position as a News Applications Developer in the newsroom of the Chicago Tribune I have received many questions from friends and family about the nature and efficacy of the work I will be doing.  Although it is straightforward to summarize the responsibilities it is difficult for me to explain why making applications out of news is such a good idea and how it contributes to the broader theme of saving journalism from decline.  In an effort to explain myself and to prepare mentally for this new endeavor I have written the following essay.

Please note: This is a statement of my opinions. It is not an analysis of the policies of the Chicago Tribune, nor is it a prescription for them.
 


 
journalism: the collection and editing of news for presentation through the media [1]
news: a report of recent events [2]
 
 

PROLOGUE

Journalism is in a period of introspection.  Thanks to a headlong decline in newspaper subscriptions, a gutting loss of revenue amongst major media outlets and the apparent political polarization of professional reporting there is an unprecedented outpouring of thought, conversation, and writing about the future of serious news reporting.  These trends, combined with the impact of digital media, have washed away years of hegemony and entrenched ideologies amongst both journalists and the consumers of their craft.  The result is a second adolescence for one of the world’s most recognized industries—and a corresponding flowering of nervous experimentation.  One need only be passingly interested in this topic to have felt the impact of new media outlets such as Huffington Post or to have noticed oneself being distracted from a newspaper’s front page by the writing of a blogger who has neither a press credential nor big media paycheck.  These changes are not killing journalism, but they are mercilessly dragging it into the strange new world of instant communication that leaders in all industries are struggling to understand.  Though this is often characterized as a tragedy, no one should be better prepared for such change and struggle than journalists are.  If this is treated as an opportunity to renew the forms and functions of their craft, this generation may see more significant and permanent contributions made by journalists than any before it.
 
 

NEWS APPLICATIONS

News applications are the offspring of a frantic love affair between journalists and the first generation of software hackers who planted their feet outside of Silicon Valley.  This union has been a long-time coming.  Both of these groups subscribe to a fast-and-loose “rebel rebel” cynicism about their craft and both are prone to doing things the wrong way and having it work out well.  For years they were held back by an old media culture that thought its history provided it with insulation from change.  This mindset was, rather predictably, wrong.  Everywhere you look journalism is changing to take on the forms of new technology.  It is becoming interactive, visual and immediate.  Sometimes it looks quite a bit like the journalism everyone is used to, such as the New York Times’ interactive Election Results 2008 map, but often it looks like something else entirely—EveryBlock, This We Know or Google Finance.  The latter sites do not have bylines, stories or the mixture of content that we expect from a media outlet, but nevertheless they provide us with useful, current information that we would not have had access to by asking our neighbors.  By verdict of common sense, as well as the definitions cited above, this constitutes news.  These sites, and those like them, characterize a new type of journalism—one that is driven by data, available on demand and targeted at niche audiences rather than a gestalt advertising bloc.  These are news applications and they are a crucial part of the future establishment of journalism.
 
 

BETTER JOURNALISM

Journalists are, by the very factual nature of their process, gathering data and such data has value aside from its role in the story that the journalist is actively producing.  As Adrian Holovaty, creator EveryBlock, has said,

So much of what local journalists collect day-to-day is structured information: the type of information that can be sliced-and-diced, in an automated fashion, by computers. Yet the information gets distilled into a big blob of text — a newspaper story — that has no chance of being repurposed.” [3]

As an example, the who, what, where, when, why and how of a high-rise apartment building fire are undoubtedly interesting as a subject of exposition, but they could also be elements of a larger story about the types and causes of fires in a city.  As words in a prose story the individual facts are disconnected from one another, but if correctly organized and segmented they can be analyzed with the larger picture in mind.  Imagine, for instance, having the ability to cross-reference the addresses of ten years worth of such fires with the names of the construction companies who erected the buildings.  A single fire is a story, but a higher than normal percentage of fires in the buildings of a single company may be an indicator of a public safety crisis.

Newspapers are ideally positioned to acquire, aggregate and analyze this sort of data.  Even internet prognosticator Robert Scoble has identified “Systems for aggregating and archiving information” as one of a dozen valuable assets that the press needs to leverage to stay relevant. [4] By formalizing the storage and representation of this information the newspaper gains all the obvious opportunities for uncovering “scoops” as well as a leg-up on the competition whenever they have to write a story about a fire or perform an investigation into public safety or answer a question about efficacy of regional fire departments.  This might be called the “archival advantage” of a news application.  It provides working benefits to the journalists and the improves the quality of their reporting.

When the data itself is published the value becomes even more significant.  Allowing a reader to understand how the issues relate to them in particular makes it something of real and practical value for them.  Consider, for example, the Nursing Home Safety Report produced by the Chicago Tribune.  This application allows users to research the Medicare ratings and safety records of any nursing home in the state of Illinois.  Instead of being limited to a story that discusses only the grossest safety violations a reader has access to all the details for the particular nursing home that matters to them.  The journalists, by becoming the curators of a well-sculpted data set, are thus able to provide more actionable information to their readers than they ever could with the narrative form and they can do so in a format that is bookmarkable, linkable, e-mailable, tweetable and generally more appropriate to the bite-sized sensibilities of a web-bound audience.  This is not to imply that prose should take a backseat to the application.  The “story” is the analysis and that provides the context for the data, but by making the data otherwise freestanding it can be sliced, diced and interrogated as the reader sees fit.  This might be called the “library advantage” of a news application—readers can access exactly the information they need.

Even stories that do not justify a complete and freestanding news application can benefit from the formalization of data via the production of static or interactive visualizations.  As Chicago journalist Steve Rhodes has written, “you can do what reporters are always taught to do but rarely pull off: Show, don’t tell.” [5] One need only examine the frequently brilliant examples posted on the blog Information Is Beautiful to see how presenting complex data using illustrations can make it more palatable to the non-expert readers.  Or consider the New York Times Immigration Explorer—a stunning visualization with a modest amount of interactivity.  This sort of presentation of the facts is impossible without the organization of the necessary data sets.

The application of library-like structure to the facts of the news might seem like an impossible task to assign to already overworked journalists.  But in reality this sort of inventorying need not be onerous.  It may take the form of anything from a rigorous hierarchical index to ad hoc tagging to completely automated semantic analysis such as is made possible by OpenCalais.  Any of these can give structure to otherwise arbitrary information and provide advantages for both journalists and readers.  As an extreme example, the New York Times is in the process of publishing much of its news content as linked data—a format which will allow applications to connect the Times’ stories up with other data on the internet via a shared language of descriptive tags. [6] This has real and practical benefits for the Times’ writers as they will now be able to query over their own journalistic history as well as any other linked data published by third parties.  This is, in effect, automating research.  Whats more, this interconnected web of data provides innumerable links back to the Times’ content, which increases their traffic, strengthens their brand and broadens their advertising base.
 
 

NEW REVENUE

Ryan Sholin has said it best, “the broken business model of newspapers remains the giant elephant in the room.” [7] Big-ticket advertising can not be relied upon and no magic bullet seems forthcoming.  However, one trend that shows promise is the steady migration toward niche-oriented content that has been successful for a variety of online businesses.  Unlike the general public, niche audiences have shown a willingness to pay for content that is suitably aligned with their expectations.  The Wall Street Journal is famous for being the only major media outlet to successfully transition its content behind a pay wall.  They accomplished this by providing a blend of propriety financial information and expert analysis that could not be found elsewhere.  For the readers of The Wall Street Journal this content had value that warranted a subscription fee.  The same is true for any niche.  Per Justin Pontin, Editor in Chief of Technology Review, “Editors can charge readers for content that is uniquely intelligent; that relies on proprietary data, investigation, or analysis; that helps readers with their jobs, investments, or personal consumption; or that is very expensively designed.” [8] To his list I would also add the prevention of physical or emotional pain.

This suggests that there are new revenue sources to found by applying niche-oriented thinking to the news and to applications in particular.  As each news application is a microcosm it is possible to leverage the same sort of value that supports exclusive financial publications, albeit on a smaller scale.  This could be done with pay walls or a “freemium” model, but obviously this will not be appropriate in every case.  In particular, applications related to breaking stories in health, politics, etc. are almost certainly required to be free for the sake of the public interest.  Moreover, an application would have to be of a suitable scale in order to justify readers paying for access.  Still, some applications with on-going value could be spun off and developed into dedicated properties.  More to the point, an application need not be behind a pay wall to be lucrative—the focused nature of their content makes them particularly valuable for targeted advertising.  For example, there is little doubt that construction companies would pay very well for advertising that consistently runs alongside a news application related to fire safety.  So long as such deals can be struck while avoiding conflicts of interest, this seems like a particularly good opportunity for publishers to earn back some of the revenue being lost in broad spectrum advertising.
 
 

NEW CREDIBILITY

Craig Newmark, founder of Craigslist, has written that “Trust is the new black”. [9] His sentiment implies a prescription for media companies; namely, that they should focus on rebuilding their credibility in a era when networking, not the bully pulpit, is the currency of reputation.  Due in large part to the distributed nature of the internet modern readers have developed a tendency to assign authority based personal experience and the weight of social relationships rather than prestige.  Social networks, such as Twitter and Facebook are now responsible for a unprecedented amount of content referral, including breaking news, expert opinions, and product recommendations.  This shift in distribution has undermined the historically necessary position of the press.

These new readers acquire their content from many sources and each may be write about nothing but a single concern that they have.  Their interest in having one provider for everything from Washington politics to gardening tips has been eroded and stands to be lost entirely.  The internet is, quite simply, too convenient for anyone to care about consolidation.  This implies that those journalistic concerns that want to survive the sea change must do so by reframing the presentation of their content so that it can more easily appeal to niche audiences.  The most obvious way to accomplish this goal is to focus on being the authoritative source for local content.  There will always be a multitude of sources for national and global news, but city and state-oriented news providers have an opportunity to acquire the dedication of a less competed for and more stable audience.  Moreover, resources developed for these smaller markets are less likely to have immediate competition and therefore more likely to become relied-upon by their users.

News applications, by there very nature, provide an opportunity to focus on smaller audiences, to build out new online properties, and to become the definitive resources for narrow topics.  All of these increase credibility amongst networked groups.  Each application provides an opportunity for branding and driving traffic back to a central website or other ad-supported properties.  Also, as a side benefit, these narrower properties are much easier to market and optimize for search ranking. For a community or regionally focused newspaper this might mean that a reader regularly visits four or five different sites, but strongly identifies each with the brand of the newspaper. Just as publisher’s might have several print publications so should they have various online properties and leverage the connections between them.  Readers will pay for that benefit with links and referrals.  The end result is new, reader-driven credibility in the only growing market for news.
 
 

NEWS APPLICATIONS II

News applications provide an opportunity to produce better journalism, to do it sustainably, and to leverage new online modes of distributing authority.  That being said, those advantages should not mislead anyone into believing that these eager offspring of hackers and journalists are anything resembling a panacea for the problems of journalism.  More and more media content is inevitably going to be produced this way, but it is just one part of a long list of new experiments that are going to contribute the renewal of the craft.  Blogging and pro-am cooperatives, e-readers and other new distribution methods, and non-profit funding models are just a few other ideas that are bound to play a role in preventing the actualization of what Jason Pontin has described as “the Götterdämmerung-of-mainstream-media argument.” [10] No one thing is going to bring stability back to journalism, but it is evident that of all the things newspapers and media companies could try news applications are a rare investment that is likely to yield both expanded revenue opportunities and better journalism.
 
 
Sources

  1. http://www.merriam-webster.com/dictionary/journalism
  2. http://www.merriam-webster.com/dictionary/news
  3. http://www.holovaty.com/writing/fundamental-change/
  4. http://scobleizer.com/2009/04/19/the-newspaper-industry-just-gave-away-another-free-meal-er-twitter-do-they-have-any-left/
  5. http://www.beachwoodreporter.com/column/the_monday_papers_142.php
  6. http://open.blogs.nytimes.com/2009/06/26/nyt-to-release-thesaurus-and-enter-linked-data-cloud/
  7. http://ryansholin.com/2008/07/24/the-business-model-is-still-the-elephant-in-the-room/
  8. http://www.technologyreview.com/blog/pontin/23489/
  9. http://www.huffingtonpost.com/craig-newmark/a-nerds-take-on-the-futur_b_325544.html
  10. http://www.technologyreview.com/blog/pontin/23489/

Non-Relational Databases Presentation

Here are the slides for the presentation I delivered at last night’s San Luis Obispo .NET User Group. I really enjoyed doing this one and it seemed to get a solid response. Many thanks to everyone who attended to kept me on my toes.


I will not blog about blogging.

But it is worth saying that I am spending much more time on Twitter these days. There I can ramble knowingly about open source, government, music, games, and assorted conversational memes all while having my tongue very appropriately viced at the 140 character mark. It seems a worthwhile limitation to me.

For documentary purposes I have added a page on Voter’s Daily with links out to the development hubs.

I have a talk on CouchDB coming up very shortly. I will post my slides here once it has taken place.

Incidentally, despite not being very active on here of late, my page rank is way up: I now beat Wikipedia’s entry on “Bouvard et Pechucet” for bouvard And I’ve knocked out the top ten for my name. I find this amusing in a way that I have not managed to put my finger on.


Love and Government Data Manipulation

I really need to publish some haphazardly swirling thoughts. In order to give order to them I am going to mend myself an a exposition wherein I simultaneously send some due praise toward my wife and also discuss my newest programming project. Here goes.

My wife, Camilla, has had to put up with an abundance of changing attitudes from me in the almost-six years we have known one another. If my recent recuperative blog posts weren’t enough indication; I’m rather prone to flights of fancy. Said flights tend to manifest in the form of rather abrupt mood swings, especially when I overextend myself. I have been making a very conscious effort to maintain a better balance in my life of late and I think I have actually been succeeding. From an interior perspective, I feel much better. A large part of this has come from giving up on some unreasonable expectations that I had for myself. This scares my wife, because it follows a pattern I have long suffered through wherein I feast on aspiration and subsequently starve on disappointment. So, I’m sure it comes as an even greater worry to her to know that I have started another big, complex project:

Voter’s Daily: an attempt to index government schedules into a consistent format.

This is a really large project–the sort of project that has gotten me into trouble before. My wife has shown great patience for my inconstancy, but I think we both know that my grand schemes are a constant test. So I believe I owe it to this woman who has seen me through so many fits of discontent, to justify why I believe I can start this now without it consuming me. Here are my reasons:

  • This project is fundamentally charitable. I can never make money off this, so this can never become about saving us.
  • This project is large, but not unprecedented, and falls entirely within my existing skill set.
  • This project can be tackled in reasonably sized chunks and will not “fail to build”, because I decided to rewrite the URL concatenation routine at ten thirty on a Sunday.
  • This project reminds me of and inspires me toward a better way to be.
  • This project has already generated more interest from a peer group that I respect, than any I have ever worked on.
  • This project is public from day one. If I ever need to leave it, it will not be defined by my absence.

On the first night after starting this project, I skipped writing code to watch a movie with my wife. I am going to be doing that a lot. I am going to skip coding to spend time with my son, to clean up after the dog, and to the pay bills. I am not going to be disappointed that this does not all come together overnight. If anyone following the project expects that it will, then I suggest that they fork it and do the work. I hope that they will and that those contributions will make it a stronger project.

I also hope that it will live up to the hype, because I think its an fantastic idea and I would not be getting fired up about it otherwise.

But more importantly, I hope that this explanation will remind my wife, via methods both diffuse and pointed, that indeed we are both awesome, but that she is infinitely more awesome for putting up with me. I hope she will come to believe that we can once again safely aspire toward great things and spite our limitations. In short, I want us to live up to the hype, too.

I love you, babe.

Commit. Push.


My New Leisure

Do not take the title of this entry too seriously. I am not suddenly awash in wine and sunshine anything. However, I am happy to report that since my conscious resolution to make some behavioral changes I have actually done so. Here are some things I am actually doing:

  • Reading Moby Dick
  • Reading The Pleasures and Sorrows of Work
  • Experimenting with Adobe Flex and AIR (thanks for free offer here)
  • Working on an artistic project that I thought of years ago
  • Taking my son to the Obon Festival at the local Buddhist Temple on Saturday
  • Migrating the Onyxfish information to a sub-section of this site

However, there is a coordinating flip side. Since deciding that profitable code is no longer my raison d’être I have been mentally inundated with an impossible surge of things that I would like to be spending time on. Of course, this is utterly contrary to what I am trying to accomplish. Being frustrated because I do not have time for altruistic things is no better than being frustrated because I do not have time for profitable things. It is going to take significant effort to stifle these internal pressures. For the sake of record, here are some of things I would enjoy working on right now:

  • Aforementioned art project
  • Re-firing NoStaples
  • Starting the companion project to NoStaples
  • Entering the GitHub contest
  • Working my way through Natural Language Processing with Python
  • Contributing to some of the fabulous projects at Sunlight Labs
  • Contributing IT work to some local non-profits and/or schools
  • Continuing development on my web application (”Rockgarden”)
  • Reading more
  • Gaming more

Inhale. Exhale. Inhale. Exhale.