tonybaldwin | blog

non compos mentis

Archive for the ‘baldwinsoftware’ Category

Baldwin Web Services

leave a comment »

I redesigned my webmaster services site today.

baldwin web services

myownsite.me

I installed dokuwiki, in truth, then modified the mmclean template, creating the DarkClean Template, to fit better with the remainder of my site.

I rewrote some of the content, stripping a lot of it down (especially the index page), and have all content in both English and Spanish. I think simplifying matters as I have will make the site more accessible to potential clients. Maybe this will help me get some webmaster work, since translation work, lately, has been sporadic and/or offered at abysmal rates.

Advertisements

Written by tonybaldwin

September 21, 2012 at 7:41 pm

Posted in baldwinsoftware

Preview of Xpostulate Improvements

leave a comment »

A preview of what’s to come…

Thinking of UI enhancements, I added the Xpostulate little icon thingy right into the GUI.

What do you think?

Other items on their way:

  • Posterous support. I have interacted with the posterous api via bash with curl, so, just need to translate my scripting for that to tcl with http. Cake, but requires time. I thought I would have that done this past week, but, no joy…too much work (somebody’s gotta pay the rent around here).
  • Blogger support. – The great and benevolent Google® has granted me an API key, and I have looked at the API, but not yet played with it, but this is likely to come this season…soon, me dro0gies.
  • Read your statusnet public timeline or updates from a specific person. This I have, again, done in bash, so just a matter of coding it into tcl. Although, I question if this is appropriate for Xpostulate, and whether it might not be better to do this with iDenTickles only, since iDenTickles is a microblogging client, and Xpostulate is intended for crossposting to blogs, not reading others’ updates.
  • Download, edit, & republish older entries. This is on my todo list, but for each blogging service I have to look at how their API handles this, and then code stuff in, and develop new GUI elements for housing various functions, and blah, blah, blah. It will be work..heavy lifting…but it’s on my TODO list.

posted with Xpostulate

Written by tonybaldwin

September 25, 2011 at 5:25 am

Posted in baldwinsoftware

Preview of Xpostulate Improvements

leave a comment »

A preview of what’s to come…

Thinking of UI enhancements, I added the Xpostulate little icon thingy right into the GUI.

What do you think?

Other items on their way:

  • Posterous support. I have interacted with the posterous api via bash with curl, so, just need to translate my scripting for that to tcl with http. Cake, but requires time. I thought I would have that done this past week, but, no joy…too much work (somebody’s gotta pay the rent around here).
  • Blogger support. – The great and benevolent Google® has granted me an API key, and I have looked at the API, but not yet played with it, but this is likely to come this season…soon, me dro0gies.
  • Read your statusnet public timeline or updates from a specific person. This I have, again, done in bash, so just a matter of coding it into tcl. Although, I question if this is appropriate for Xpostulate, and whether it might not be better to do this with iDenTickles only, since iDenTickles is a microblogging client, and Xpostulate is intended for crossposting to blogs, not reading others’ updates.
  • Download, edit, & republish older entries. This is on my todo list, but for each blogging service I have to look at how their API handles this, and then code stuff in, and develop new GUI elements for housing various functions, and blah, blah, blah. It will be work..heavy lifting…but it’s on my TODO list.

posted with Xpostulate

Written by tonybaldwin

September 24, 2011 at 10:16 pm

Web Word Count

leave a comment »

Web Word Count: Get the word count for a list of webpages on a website.

A colleague asked what the easiest way was to get the word count for a list of pages on a website (for estimation purposes for a translation project).

This is what I came up with:

#!/bin/bash
# add up wordcounts for website

total=0 # initialize variable for total

# scan through a list of pages
# strip the html elements and count the words
# append the count to wordcount.txt

for i in $(cat pagelist.txt); do
     curl -s  $i |  sed -ne '{s/]*>//g;s/^[ \t]*//;p}' | wc -w >> wordcount.txt
done

# this is for purely aesthetic purposes, 
# but we're merging the list of pages with the wordcount file:
paste pagelist.txt wordcount.txt > pagewordcount.list

# for each number in the wordcount.txt file, add it to the previous number (get a total)
for t in $(cat wordcount.txt); do 
	total=$((total + $t))
done

# append the total to the end of the merged pagelist+wordcount file:
echo "Total word count = $total" >> pagewordcount.list

# read back the file:
cat pagewordcount.list

# ciao
exit

I ssh-ed to my server and did
ls -1 *.html > pagelist.txt
which lallowed me to feed the script this list.

baldwinlinguas.com/index.html
baldwinlinguas.com/esp.html
baldwinlinguas.com/fran.html
baldwinlinguas.com/port.html
baldwinlinguas.com/empregos.html
baldwinlinguas.com/transquote.html

So, then I ran the script on this list of the pages, and this is the output:

baldwinlinguas.com/index.html 535
baldwinlinguas.com/esp.html 342
baldwinlinguas.com/fran.html 295
baldwinlinguas.com/port.html 337
baldwinlinguas.com/empregos.html 662
baldwinlinguas.com/transquote.html 244
Total word count = 2415

So, it works. Someone with better bash fu could likely find a shorter path to this result.

Now, this is simple, of course, for a simple website, like baldwinlinguas.com.
On the other hand, if you have some huge wordpress installation, like this blog, and have tonso public php pages, rather than html, and eve more php files in the backend, you have to do a bit of sorting, I imagine.

Were I to attempt that with the baldwinsoftware wiki, I would probably just go to the Sitemap and grab that list of pages, using their URLs, of course.

./tony

Written by tonybaldwin

September 21, 2011 at 5:25 am

Web Word Count – count the words on a website with bash, lynx, curl, wget, sed, and wc

with one comment

Web Word Count: Get the word count for a list of webpages on a website.

A colleague asked what the easiest way was to get the word count for a list of pages on a website (for estimation purposes for a translation project).

This is what I came up with:

#!/bin/bash

# get word counts and generate estimated price for localization of a website
# by tony baldwin / baldwinsoftware.com
# with help from the linuxfortranslators group on yahoo!
# released according to the terms of the Gnu Publi License, v. 3 or later

# collecting necessary data:
read -p "Please enter the per word rate (only numbers, like 0.12): " rate
read -p "Enter currency (letters only, EU, USD, etc.): " cur
read -p "Enter domain (do not include http://www, just, for example, somedomain.com): " url

# if we've run this script in this dir, old files will mess us up
for i in pagelist.txt wordcount.txt plist-wcount.txt; do
	if [[ -f $i ]]; then
		echo removing old $i
		rm $i
	fi
done

echo "getting pages ...  this could take a bit ... "

wget -m -q -E -R jpg,tar,gz,png,gif,mpg,mp3,iso,wav,ogg,ogv,css,zip,djvu,js,rar,mov,3gp,tiff,mng $url
find . -type f | grep html > pagelist.txt

echo "okay, counting words...yeah...we're counting words..."

for file in $(cat pagelist.txt); do
	lynx -dump -nolist  $file | wc -w >> wordcount.txt
done
paste pagelist.txt wordcount.txt > plist-wcount.txt

echo "adding up totals...almost there..."
total=0
for t in $(cat wordcount.txt); do
	total=$((total + t))
done

echo "calculating price ... "
price=`echo "$total * $rate" | bc`

echo -e "\n-------------------------------\nTOTAL WORD COUNT = $total" >> plist-wcount.txt
echo -e "at $rate, the estimated price is $cur $price
------------------------------" >> plist-wcount.txt

echo "Okay, that should just about do it!"
echo  -------------------------------
sed 's/\.\///g' plist-wcount.txt > $url.estimate.txt
rm plist-wcount.txt
cat $url.estimate.txt
echo This information is saved in $url.estimate.txt
exit

So, then I ran the script on my site, tonybaldwin.net, with a rate of US$012/word, and this is the final output:

—————————————-
tonybaldwin.net/log/archives/environment/index.html 38
tonybaldwin.net/log/archives/cuisine/index.html 38
tonybaldwin.net/log/archives/music/index.html 52
tonybaldwin.net/log/archives/philosophy/index.html 38
tonybaldwin.net/log/archives/nanoblogger-help/index.html 52
tonybaldwin.net/log/archives/2011/09/11/911/index.html 322
tonybaldwin.net/log/archives/2011/09/index.html 774
tonybaldwin.net/log/archives/2011/09/01/mit_intro_to_cs_and_programming_assignment_1/index.html 494
tonybaldwin.net/log/archives/2011/08/26/come_on_irene/index.html 382
tonybaldwin.net/log/archives/2011/08/26/welcome_to_nanoblogger_3_4_2/index.html 289
tonybaldwin.net/log/archives/2011/08/26/here_we_roll_again/index.html 618
tonybaldwin.net/log/archives/2011/08/27/couldnt_stand_the_weather/index.html 93
tonybaldwin.net/log/archives/2011/08/index.html 1205
tonybaldwin.net/log/archives/2011/index.html 133
tonybaldwin.net/log/archives/technology/index.html 56
tonybaldwin.net/log/archives/politic/index.html 38
tonybaldwin.net/log/archives/religion/index.html 38
tonybaldwin.net/log/archives/art/index.html 38
tonybaldwin.net/log/archives/index.html 85
tonybaldwin.net/log/archives/personal/index.html 65
tonybaldwin.net/log/archives/health/index.html 38
tonybaldwin.net/log/articles/about/index.html 671
tonybaldwin.net/log/index.html 2027
tonybaldwin.net/log.1.html 2027
tonybaldwin.net/index.html 96
tonybaldwin.net/social.html 82

———————————————–
TOTAL WORD COUNT = 9789
at 0.12, the estimated price is USD 1174.68
———————————————–

Now, this is simple, of course, for a simple website, like tonybaldwin.net, which is largely all static html pages. Sites with dynamic content are going to be an entirely different story, of course.

The comments explain what’s going on here, but I explain in greater detail here on the baldwinsoftware wiki.

Now, if you just want the wordcount for one page, try this:

    #!/bin/bash

# add up wordcounts for one webpage

if [[ ! $* ]]; then
    read -p "Please enter a webpage url: " ur
else
    url=$*
 fi
 read -p "How much to you charge per word? " rate
 count=`lynx -dump -nolist $url | wc -w`
 price=`echo "$count * $rate" | bc`
 echo -e "$url has $count words. At $rate, the price would be US\$$price."
 exit

Special thanks to out to the Linux 4 Translator list for some assistance with this script.

Enjoy!

./tony

Written by tonybaldwin

September 20, 2011 at 10:31 pm

Post to Posterous with bash, curl and vim

leave a comment »

Here we have a nifty little script to post to posterous.com from the bash terminal with vim and curl.

#!/bin/bash
# post to posterous from bash cli with curl
# I could do this with read -p "Enter ur username d0od: " username
# kind of thing, but I just hardwired my info in.
# edit accordingly for your info.
username=YOUR_USERNAME
password=YOUR_PASSWORD
apitok=YOUR_API_TOKEN

# creates a date stamp for naming the post file
filedate=$(date %m%d%y%H%M%S)

# set post title
read -p "Enter a post title: " ptitle

# write post in vim
vim $filedate.ppost pbody="$(cat $filedate.ppost)"

# send post to posterous with curl
if [[ $(curl -X POST -u $username:$password -d "api_token=$apitok" -d "post[title]=$ptitle" -d "post[body]=$pbody" http://posterous.com/api/2/sites/1814977/posts | grep error) ]]; then
echo "Too bad, do0d...FAIL!"
else
echo "Success! Posted to Posterous!"
fi

# moved post to dir for safekeeping.
# you can use different dir, d00d
mv $(pwd)/$filedate.ppost ~/Documents/fposts/

# open a browser for a look-see?
read -p "Shall we have a look in your browser now? (y/n): " browse

if [ $browse = "y" ]; then
	xdg-open http://$username.posterous.com/
fi

exit

Here is first post made with this script to my posterious. In truth, however, I do not use posterous, really. I will be using this information to add posterous support to Xpostulate before the week is out, however, per a user request. The experimentation with curl was to get my head around their API and posting procedure.

Here is the relevant wiki page for this script.

enjoy…

./tony


posted with Xpostulate.

Written by tonybaldwin

September 19, 2011 at 9:46 pm

Posted in baldwinsoftware

Posting to Posterous with curl

leave a comment »

Okay, so I DID successfully post with curl (thanks to some clarifications frmo John Tucker). Now, I have written a script that will allow me to write a post in vim and fire it off. It looks like this:

#!/bin/bash 
# post to posterous from bash cli with curl 

# I could do this with read -p "Enter ur username d0od: " username 
# kind of thing, but I just hardwired my info in. 
# edit accordingly for your info. 
username=YOUR_USERNAME 
password=YOUR_PASSWORD 
apitok=YOUR_API_TOKEN 

# creates a date stamp for naming the post file 
filedate=$(date %m%d%y%H%M%S) 

# set 
post title read -p "Enter a post title: " ptitle 

# write post in vim 
vim $filedate.ppost pbody="$(cat $filedate.ppost)" 

# send post to posterous with curl 

if [[ $(curl -X POST -u $username:$password -d "api_token=$apitok" \
-d "post[title]=$ptitle" -d "post[body]=$pbody" \
http://posterous.com/api/2/sites/1814977/posts | grep error) ]]; then 
echo "Too bad, do0d...FAIL!" 
else 
echo "Success! Posted to Posterous!" 
fi 

mv $(pwd)/$filedate.ppost ~/Documents/fposts/ 
# moved post to dir for safekeeping. 
# you can use different dir, d00d 

exit

This means that I have consquered the posterous API, and will now be able to add posterous support to Xpostulate.

Written by tonybaldwin

September 19, 2011 at 6:48 pm