Mining the Ngram Viewer

Google Books Ngram Viewer is a nifty tool that analyzes all the text of all the books Google has digitized (over 25 million and counting) and lets you see the relative frequency of words going back to the 1600s.

What isn’t immediately obvious to most people is what you can do with Ngram Viewer — what kinds of insights you can glean from analyzing the text within books. I don’t have an easy answer, but here are a few ways to search Ngram Viewer. Leave a comment and let me know what you’ve been able to do with this intriguing research tool.

Compare the relative popularity of concepts over time.  For example, you can compare the frequency of the words progress, tradition and innovation over the decades. To make this more intriguing, note the different results when you limit your search to British English or American English.

Search for any word that appears near a specific word. If, example, you were researching the nursing profession, you might want to see what words most commonly precede the noun “nurse”. Using the syntax *_NOUN nurse, you can note the spike and subsequent drop in frequency of “head nurse” in the middle of the 1900s.

Compare the prevalence of a concept in fiction against all English-language text. If you want to see how frequently doctors and nurses are mentioned in fiction, the query nurse:eng_2012,nurse:eng_fiction_2012,doctor:eng_2012,doctor:eng_fiction_2012 will show you that doctors get the most press.

Granted, the syntax requires you to channel your inner programming nerd, and it takes some creativity to figure out how to use the Ngram Viewer. If you want to dig even deeper into all of its capabilities, check out the advanced search page.

Is Google feeding confirmation bias?

snippetIn its ongoing effort to answer the world’s questions (and sell ads), Google has been putting increased emphasis on its “featured snippets” – the little boxes of text extracted from whatever source Google has calculated to be most relevant. If I want to see whether my dogs can catch the flu, I can quickly see that, yes, it’s possible.

However, a recent Wall Street Journal article (“Google Has Picked an Answer for You—Too Bad It’s Often Wrong“) looked at the increased frequency of these quick answers that appear at the top of search results. (Note that these are not the Knowledge Panels, which are sourced from Wikipedia and other neutral sources.)

According to a study commissioned by the WSJ, these featured snippets are often excerpted from unreliable or biased sources. Google’s algorithm favors a web site with text that most exactly matches the query; as a result, the researchers found that the extracted text was more likely to come from a less-authoritative, biased or dodgy clickbait site.

Worse yet, since featured snippets are designed to closely match the query, they can feed confirmation bias. The featured snippet for the query “is milk good for you” says “Milk can be good for the bones because it provides vitamin D and calcium…” The featured snippet for the search “is milk bad for you” says “Animal milk has long been claimed as the go-to source of calcium by the dairy industry, but as it turns out, milk is bad for you .”

Since most people have been trained by Google to trust the first answer that appears, it’s even more important to practice some information hygiene before relying on the first answer from a search engine.

Do you trust that news?

skeptical woman looking over her glassesIn its ongoing efforts to address the scourge of misleading and false news, Google recently announced a new feature that helps readers evaluate a news source they may not be familiar with. Now,  when you search for a particular publication, the Knowledge Panel – that preformatted answers box that often appears at the top of search results – includes information about that publisher.

Depending on the publication, that can include awards they have won, the topics they cover most extensively and their political alignment. If content from the publication has recently been reviewed by an authoritative fact-checker, those items are also featured in the Knowledge Panel. [UPDATED: This seems to work in Google Chrome and Safari, but not Firefox. Thanks, Pam Wren, for the heads up!]

So, for example, if you Google “Wall Street Journal”, your search results page will include a Knowledge Panel like this:


You’ll see a one-sentence blurb from the Wikipedia article about the newspaper, links to professional awards for reporting, and a summary of the topics they have recently covered — in the case of the Wall Street Journal, that’s the Federal Reserve, advertising, sales and taxes… about right for a newspaper described as business-focused.

And if you Google “Breitbart”, your search results page will include a Knowledge Panel like this:


If you click the link for “Writes About”, you’ll see that Breitbart has recently covered Donald Trump, Barack Obama, the Republican Party and Hillary Clinton… what you might expect from what the Wikipedia article describes as a “far-right American news, opinion and commentary website”. But note the “Reviewed Claims” tab, highlighting reported facts that were then determined to be false by fact-checkers like Snopes, Politifact and FactCheck. This stands out as a concern — most news sources’ Knowledge Panels don’t include lists of reported facts that were questioned and reviewed by fact-checking sites.

This is a great way for librarians and information professionals to instill a little FUD (Fear, Uncertainty and Doubt) when their clients assume that whatever they see on their Facebook feed is reliable. And check out Vanessa Otero’s infographic, What, Exactly, Are We Reading?,  a nice chart of where various media sources fall, both in terms of reliability/fabrication and liberal/conservative.

For Google, it’s location, location, location

I know… all Google is trying to do is help you get “better”, or at least more relevant, results from a search. And Google has assumed that you are your location — that where you are searching from really matters. Much of the time, that’s great. But for us professional searchers who search outside our own country, Google has just made a change that will significantly affect our search strategies.

Until now, if you wanted to focus your search on results from the UK and you were located in the US, you would go to the UK version of Google at And yes, you’d always get different results than when you ran the identical search in However, according to a recent Google blog post, this trick will no longer work.

Now the choice of country service will no longer be indicated by domain. Instead, by default, you’ll be served the country service that corresponds to your location. So if you live in Australia, you’ll automatically receive the country service for Australia, but when you travel to New Zealand, your results will switch automatically to the country service for New Zealand. Upon return to Australia, you will seamlessly revert back to the Australian country service.

There’s a workaround; go to Settings and select Advanced Search. Scroll down to “Then narrow your results by…”, pull down the Region menu, and select the country you want to use to focus your search.

step-1       step-2

I tried this out with a search for Brexit, first searching in, then in with the Region set as United Kingdom. And, just curious to see if it would make a difference, I tried a third search in after setting my VPN to connect in the UK. And I got different search results from all three searches. Below are the top search results, highlighting the results that only showed up at the top of one of the three searches. Note that each search turned up results that weren’t in the top of the other two.


Bottom line: I’ll now be doing three searches when I’m using Google to find information from a specific country or region. Thanks, Google…


Super searching tips

I just got back from Internet Librarian 2017 (in beautiful Monterey, CA — tough assignment). Among the insights I’ve brought back are:

Google Image search is focused more on matching meaning than matching images. If you want to search for instances of an image (to watch for usage of your organization’s images or to find mentions of a chart or graph in a report or article, say), you’re better off using a reverse-image search tool like TinEye instead.

A use of reverse image search I don’t often remember is to see if you’re looking a legitimate profile in social media or a fake. Right-click the person’s image, copy the URL and search for other instances of that image. If it’s a fake profile, it’s likely that whoever set up the profile used an image that appears elsewhere on the web, often a stock photo.

Remember Google’s undocumented (i.e., not in Google Help) prefix searches.  You can use intext: to look for words in the body of the page, intitle: for words in the title, inurl: for words that appear in the URL itself; and inanchor: for the words that appear in the anchor text (the text that’s highlighted in a hyperlink). Remember that you can’t have a space between the prefix and your search term — use intitle:asteroid to find web pages that have the word asteroid in the title, for example.

And I just learned about a new top-level domain – .graphics, so you can look for web pages specifically pertaining to computer and data graphics by searching for site:*.graphics.

When researching a topic, consider whether you want to search by process (how do I do this activity/thing?) or outcome (how can I get this result?). You’ll use different words and find different results based on which perspective you take.

You can also see my slide decks from IL2017 — Super Searcher Strategies and (ROI) Truth to Power.

“You’re charging WHAT?!?”

Shocked and surprised childYou’ve probably had this happen to you at least once… You have what seems like a perfectly normal conversation with a prospective client, you send what you think is a perfectly reasonable proposal, and your client responds with shock at how expensive you are.

It’s easy to react badly in this kind of situation; you know how much value you bring to any project and you know you’re worth what you are charging. (If you’re not sure that you’re worth that, go read the Harvard Business Review article, “Why You Should Charge Clients More Than You Think You’re Worth,” and Mary Ellen’s Amazing Hourly Rate Calculator.)

At the beginning of my career as an solopreneur, I took this kind of response badly; I assumed that either I had vastly overrated myself and my value or I had utterly failed because no one saw what I had to offer.

Fortunately, I’ve adjusted my attitude and now have a more positive approach. When I’m told that I am too expensive, I run the Internal Entrepreneurial Translator™ in my brain and hear them saying that they simply don’t currently have the budget or need for my expertise and level of service. I also remind myself that, while they aren’t hiring me now, situations change. People change jobs, and talk with their peers. Organizations suddenly find available funds. Priorities shift. All of these are reasons to treat any rejected proposal as simply not a good match at this time, for this client, in this situation. The bottom line may just be that their organization has different priorities for their resources at this time. That’s OK; the interaction can be respectful and positive. They’re just not at a place where they need your level of service.

And I use these experiences as opportunities to look at how I’m presenting myself and my skills. I ask myself if I’m focused entirely on outcome or if I’m talking about activity. Am I focused on what my client values the most and how I can help them achieve that goal, or am I talking about what I’ll do and what my hourly rate is.

I make sure I’m not talking about how I “need” to be paid $X because I have a mortgage/rent and other expenses to pay. Because, frankly, clients don’t care about what our expenses are. All they are about is receiving high-quality services that address their needs. I focus on making sure that my price reflects the outcome my clients will see.

The Public Speaker’s Secret Weapon

Eyes Behind Red Curtains On Wood StageAlthough I’m pretty comfortable speaking in front of a crowd now, I wasn’t born that way. In fact, I remember being absolutely terrified for at least the first few dozen presentations I gave. I managed to get the terror under control but it took many years before I discovered the secret weapon that has completely turned around my experience speaking in public.

You’re probably familiar with the problem of confirmation bias – the human tendency to look for, recognize and remember information that confirms our preexisting beliefs and theories. As info pros, we have to be on guard against confirmation bias; it can blind us to relevant information that could challenge the entire premise of our research.

But for public speakers, keeping this human default setting in mind can be very reassuring. Remember, people chose to come to your presentation, join your webinar or attend the meeting you are leading. That means they all have a vested interest in confirming to themselves that they made a good decision to spend their time listening to you. They’re not secretly counting up your grammatical mistakes, judging your choice in footwear, or wondering why your hair looks like that. Rather, they are looking for evidence that they made a good choice.

Once I realized this, I developed a quick little ritual that I now practice before every presentation I give or meeting I lead. I find a quiet space (and yes, it might just be a bathroom stall), close my eyes and take a couple of deep breaths. I imagine the room filling up with people, each of whom is – subconsciously at least – rooting for me. I conjure up a feeling of gratitude that every person in the audience is on my side; they are confident that I’ll do a fine job and they know they will be glad they chose to spend time with me. I smile, even if I have to force myself, and take one more deep breath, inhaling gratitude and exhaling a calm confidence that I will show up at my very best.

Yes, it sounds woo-woo, but this technique has changed my experience of public speaking. The people who have joined me for this event are my allies, not something to be feared. They’re more than willing to forgive me when I suddenly lose my train of thought, as long as I get myself back on track and keep going. They won’t focus on whether or not I have a particular degree or so many years of experience; they just want to come away knowing more than they did before.

I still feel that adrenaline rush as I’m being introduced and I walk to the podium, but it’s not immobilizing. I recognize it as a tool to help me think on my feet, and I look at it as confirmation that I’ll give a great presentation.

Try this approach and see how it works for you. Public speaking may never feel completely natural, but you can hone your skills in managing and channeling your initial anxiety.

Killing the Task Monsters

I’m a great list-maker. I have to-do lists everywhere; they have been compiled carefully, organized strategically, color coded and tagged. But when it comes to actually getting all those listed things done, it’s another matter. Some I can get done right away, and virtuously check that item as DONE. Others I look at, think “ugh, that’s going to take time”, and skip over, day after day. Pretty soon, they become big ugly Task Monsters, glaring at me reproachfully, daring me to take them on.

I finally realized I could slay the Task Monsters the same way you eat an elephant… a bite at a time. I have gone from to-do lists to to-do index cards, and that has transformed how I approach projects both large and small. Whether it’s managing a complex client project, re-imagining a web site, developing a new marketing plan or scoping out a home renovation job, everything has its own pile of cards.

The magic is that I sort the cards based on how long I think the task will take. There are 15-minute cards for items like “Reach out to Anne to schedule informational interview” and “Email graphic designer re: logo”. There are half-hour cards for “Scope out options for new fridge” and “Sketch out upcoming webinar on big data”. For the tasks that I know will take more attention, I have 2- to 4-hour cards for jobs like “Write proposal for speaking opportunity at 2018 SLA conference” and “Outline new web page flow”.

Sure, it takes some time to chunk out all my projects into index-card-sized jobs, but I consider this time well spent. Now, when I have 20 minutes to spare before my next appointment, or a meeting was unexpectedly postponed and I have two unscheduled hours, I go through my cards to see what I feel like tackling.

The magic of these cards is that they’re so easy to pick up when you find yourself at the end of one task and not sure what to do next. While it might feel daunting to open up a folder labeled “New Marketing Plan”, it doesn’t take much to flip through some cards and choose one small item to take care of. Before you know it, you’ve made a significant dent in that daunting project. Take that, Task Monster!

ADDED: Joann Wleklinski suggested color-coding the cards – using a different color for each time category. Brilliant!

Stability vs. steady income

aesopAs a child, I read Aesop’s Fables avidly; I like getting my life lessons from animals rather than humans, I suppose. One fable that caught my attention way back then was The Dog and The Wolf, the TL;DR version of which is a well-fed dog offered to help a scrawny wolf get regular food from his master. The wolf listened but noticed a bald spot on the dog’s neck where the collar sat. Goodbye, said the wolf. There is nothing worth so much as liberty.

I often talk with coaching clients who have worked for years — often decades — as employees and who are learning how to look at the world differently. One of the biggest shifts is having to give up the idea of a steady, predictable paycheck in exchange for being self-employed. (And I’ll pause here to note that there are some situations in which a steady paycheck is more important than the liberty of self-employment. In that case, keep your day job!)

Often, new solopreneurs seek out retainer clients who will guarantee them a predictable revenue, usually in exchange for a lower hourly rate. They look for clients who, in essence, need to slot a contractor into their work process — like an employee but without the benefits or job security. They are looking for a client who looks like their last employer. (And look… there’s that bald spot on your neck, caused by the collar.)

I know what that feels like. When I launched my business, I had a retainer client who paid me for 15 hours a week at half my usual hourly rate. After six months, I realized that my nice, steady client was actually holding me back from building a stable, profitable business. I would never be able to expand my client base when so much of my time, mental energy and focus was already committed to work at an hourly rate that wasn’t going to sustain me. I took a deep breath, gave my client several months’ notice that I would need to double my hourly rate, and then helped her find someone else to take my place.

It was one of the scariest things I have done, but my next (full-hourly-rate!) client called me out of the blue the next week… and I have never looked back. I realized that having a regular stream of clients would make my business stronger and more stable than having one big client. I am better off having ten clients who use me periodically and refer me to others than having one client who could end my contract at any time. While I may not know exactly where my income will be coming from in six months, I know that, with consistent marketing and outreach efforts, clients will be calling me for projects. My business is more stable because it is based on the income from multiple sources.

Yes, there are months with thin cash flow, which I cover from the months when I am busier. I put up with the challenges that a variable cash flow present, because I’ve learned that the higher value we provide, the less likely we are to find steady clients who need to slot us into their work flow. We make a trade-off between having a predictable but lower income stream and getting paid significantly more per hour but not knowing exactly which client will be paying that high hourly rate. I don’t know about you, but I sure appreciate the extra hours in the day that I have gained by focusing on higher paying and higher value work rather than predictable but lower value work.

feed-usAnd while my own two dogs appreciate their steady and predictable paycheck of kibble every day, I’m glad that’s not how I earn my dinner.

Handling scope creep

Scope creep, the phrase that strikes fear in the heart of every consultant…

We have all had that experience, where we carefully plan out every aspect of a project, estimating the necessary time and resources and even adding in a safety margin, only to have our client ask for “just a little more” work or “just this little addition” to our deliverable halfway through the project. All of a sudden the expected work load has doubled, for no additional income.

It’s easy to react defensively when this happens — to feel that your client doesn’t respect your time or that he’s trying to get you to work at a significantly reduced fee. And it’s tempting to react angrily, to resent that the client is trying to change the agreement. (I am reminded of the immortal line in Star Wars: The Empire Strikes Back in which Darth Vader snarls, “I am altering the deal; pray I don’t alter it any further.”)

Instead, reiterate your commitment to supporting your client, find something you can say yes to and then explain where you have to say no. If your client asks for analysis or conclusions you are not comfortable making, offer to highlight the perspectives of established experts your client can rely on instead. If your client’s new request means you will spend far more hours than you had anticipated, briefly explain that this would exceed the scope of the project and offer either to do just a portion of the extra work or to write up a supplemental agreement to cover the project expansion.

While these conversations often feel stressful to solopreneurs, we are actually providing a service to our clients—helping them understand how much time, judgment and expertise is involved in what we do. A friend and colleague recently told me about the response she got from a client whose request would have required far more work than she had negotiated. After she explained the situation and offered some alternatives, her client responded with a gratifying email:

I fully understood and that is fair. You have been very generous and we can see that you have already put more time into the work than we had agreed. Anything that you may be able to get back to us that can address any of the issues we discussed would be great. Whatever you can or can’t find will be fine.

Because of my friend’s ability to calmly and politely respond to the possibility of scope creep, she continues to have a client who values her work, her time and her commitment to his concerns. He knows that her budget is honest and realistic, and that additional work involves additional commitment. She turned what could have been an unpleasant discussion into an opportunity to strengthen her relationship with her client.