Just for fun I decided to see if I could write a long Unix command line to produce a top-25 word cloud from my blog contents.

Here's what I came up with.



And here's how I generated it from the Unix command line:
cat *.txt | tr '[A-Z]' '[a-z]' | tr ' i ' ' I '| sed 's/ /\n/g' | grep ..... | \ egrep -v '[\><"#1234567890/|_)(!&-+=:@-]'| sed 's/[{}?;,.]//' | sort | uniq -c | \ sort -n | grep -v -f ~/stopwords.txt | sed 's/$/<\/font>/' | sed 's/[0-9] /">/'| \ sed 's/\([0-9]\)/<font size="\1/' | tail -25 | shuf -n 25 > ~/wordcloud.txt

In English, what those commands do are:

• List all the text files
• Transfer all the words into lower case
• Recover the word i back to I
• Substitute every space in each text file with a carriage return (now all words are on their own line)
• Apply my own "stop word" filter-- namely, only show words with 5 characters or more (not, me, us, she,... boring words for a cloud)
• Pull out lines containing non-alpha characters
• Pull off , and . and other punctuation from words
• Sort the resulting list
• Count the unique lines
• Sort the counted list
• Pull out a list of stopwords from google
• Add a closing </font> tag to the word
• Replace the space between the count and the last digit of the number with a "> to close the font size tag
• Replace the first digit(s) of the count with <font size="[number]
• Take the 25 last (most frequent) lines
• Randomize the list of 25
• Put the <font> tags into a file called wordcloud.txt

Another day, another (geeky) project. ;)