Archive for the ‘Uncategorized’ Category

that’s right, action items

Wednesday, February 4th, 2009

I am currently the dev manager for three teams.  I end up sitting in meetings much of the day and help assign and receive action items.  Generally these are not even called out as action items, but if they are, I get them (at best) in an email.  Sometimes the action items are at the end of a word doc, attached to an email.  Also know as, generally, lost.

For ages I have wanted a dumb solution and recently I put up a new site (Action-Items.com) to help out.  It is a google appengine site, and yes, rather simple.  The vision is that you would come and enter your action items, give them a priority and tags if you like, then order on priority and filter on the tags.  Rather simple, but I think even as it is, it can help solve my problem.

If you visit the site, you will notice that for now, there isn’t really anything there until you login with a google account.

Near term features include being able to assign action items with others, accept (or refuse) assigned action items, search on the full text of the action items, maybe have groups of users, and allow read only access to the items. I think that would be a pretty useful product.

Oh, and for the mrs, I plan on sometime changing the font 🙂

Feedback (feedback @ holaservers.com) would be much appreciated.

Enjoy!

Earl

flash! ugh?!

Tuesday, January 27th, 2009
yes, a viking that helps you shop

yes, a viking that helps you shop

Some few years ago, I had a D&D character named spackthar.  He ruled.  For some reason I thought it a good idea to buy a series of domain names based on the *thar theme.

  • blogthar – an awesome blogging site blogthar_logo_174x791
  • songthar – an awesome lyrics sitesongthar_logo_174x791
  • botthar – an mmo helper site
  • guildthar – an  mmo helper site that specifically helps with guilds
  • shopthar – arbitrary shopping engine

followed.  I have let most of them expire by now, but I just can’t shake loose of the shopping stuff.  Lately I have switched my efforts from holaservers back to shopping.  A few years ago I went to an amazon affiliate talk at oscon and kind of caught the vision.  I implemented the affiliate api in good old perl, and generated a few million static html pages.  I uploaded them to a variety of sites (including some of the thars above), and somehow I started selling stuff.  I started to get a thousand page views a day, and a like fifty percent traffic boost each month.  Then, I think I pushed it too far, or my ip changed, or something and I went (I think literally overnight) to like a hundred page views a day.  My sales fell off to nearly nothing, but not to nothing.  I still made like a hundred dollars a month, which was pretty cool.

Then I had a vision, and got some help.  And maybe I had the vision even when I was doing the static html, but I wanted to build an arbitrary shopping engine.  Filter on whatever you wanted and help you find the best products.  Eventually I got a little help, dove in and we made the startings of a pretty cool site.  We were using the YUI! stuff and life was good.  We started to run into some browser compatibility issues and I thought it a great idea to switch to flex.  We had made pretty everything available via a web service so I found a guy that knew flex and he just wrote around the extant web service and there we go.  Or went.  Traffic dropped to nearly nothing, or actually nothing.  Sales followed a similar trend.  There may have been a few reasons, like trying to do a little google dance, and having lame urls, but wow, talk about effort disincentive.

Lately I have decided to give it a go again.  I have implemented some sitemaps and then rewrote them to prettier urls.  I even changed the webservice so that the flex points to the pretty urls.  I have been keeping an eye on my bot traffic and it looks like something may be happening

| count(*) | date        | user_agent                                                                                 |

|       68 | 2009-01-24 | Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)                   |
|      674 | 2009-01-25 | Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)                   |
|      722 | 2009-01-26 | Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)                   |
|       30 | 2009-01-27 | Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)                   |

Those are some hits WHERE request_uri LIKE ‘/cgi/shop/product?asin=%’.  Maybe if they can get indexed well, then some real traffic will start to return.  I have also tried to have pages appear on only two domains, which is a whole lot less than the dozen or so I used to do.  A site like computercomparer has computer stuff, and then shopthar has pretty well everything.  Nothing yet, but I am still hopeful.

It is still a little rough, but if you’re looking for a computer, or a harddrive or the like, give it a go.  Even if you don’t buy I would love to start getting feedback on the forums or directly (feedback @ holaservers.com)

Enjoy!

Earl

rewriting my history

Sunday, January 18th, 2009

Recently I got this page working

http://computercomparer.com/sitemap.xml

It builds a dynamic sitemap along the lines of what I did for holaservers.  I query the database and find all the categories and products for the site.  So that happened a week or so ago.  Today I changed the product links to hopefully be more google friendly.

Before looked like this

http://computercomparer.com/cgi/shop/product?product_id=138

After looks like this

http://computercomparer.com/ap/0743251547/be+prepared+a+practical+handbook+for+new+dads

A couple I hope improvements to point out

  1. the before link is tied to my internal product id, so if I have to recreate my database, the links are hosed.
  2. the after link is tied to amazon’s asin
  3. some title text shows up in the link
  4. kind of a funny picture on the product
  5. I am actually, a relatively new dad; perhaps I should order the book
  6. that link was just the first one on my sitemap
  7. not sure why that book is in my computer category hierarchy, but there you go

Another point is that even though I plan to add more affiliates, which wouldn’t be asin based, I decided to be a little agile, and just write to what I’ve got.  I also think that once I add say buy.com, most of the stuff will likely be sold at amazon and have an asin.  Guess I need to think of a way to handle things at buy or newegg but not at amazon.  But not much more tonight.

And yes, I use mod_rewrite to go from my after link to something a little more server-friendly.

Enjoy!

Earl

I never thought these posts were real

Saturday, January 17th, 2009

Sometime ago, I got forum.holaservers.com up and running.  I never put anything on it, but today, I at least tried to have it be usable.  I don’t think anything links to it, but it is at least a start.  I added forums for HolaServers and ShopThar.  It would be nice if folks could go use the sites and come give feedback.  Especially for the shopping stuff I would folks to start to come up with user stories and then I can see if I can satisfy them.  Like cool ways to find cool products.  I am pretty sold that I can build a pretty dang useful site, but it isn’t there quite yet.

Feel free to go register and start giving feedback.

Enjoy!

Earl

I’ll link you up!

Tuesday, January 13th, 2009

Kind of afraid that folks will hate me for it, but tonight I tried to add a couple little text links to the bottom of holaservers pages.  You can see examples here and here.  I figure that so long as I don’t have banners or for sure pop-ups, then folks won’t hate it too bad, right?  Guess there’s a decent chance though.  So why risk it?

  1. Imagine a world where I have a couple million websites and since they are largely indexed, I would potentially have millions of links to cool sites like computercomparer and potentially cool sites like kindlekindle. Just can’t miss out on the potential page rank.
  2. I already wrote good old MyUrl stuff and I might as well use it.
  3. Could handle giving folks another reason to actually upgrade.
  4. It’s really not that intrusive, really.

Enjoy!

Earl

samp on OpenSolaris

Friday, January 2nd, 2009

I spent quite a bit of time getting mod_perl, apache and mysql to play nice on solaris (the nevada builds) some few months ago.  I eventually found CoolStack and after some amount of effort got up and running.  A nice private wiki page documented each command needed to get up and running on a solaris zone.  Recently I got a new X4150 from Sun and wanted to try out their project indiana stuff, which I guess is really OpenSolaris proper.  Oh, and I want my samp to run from inside a zone, did I mention that?

My goal for this post is to go from a fresh OpenSolaris 2008.11 zone and to get this one-liner to work

  • perl -e “use DBI;my \$dbh=DBI->connect(‘DBI:mysql:database=mysql’, ‘root’, ”);print ‘ping=’ . \$dbh->ping . qq{\n}”

Here is an overview

  1. Install OpenSolaris 2008.11.
  2. Create a zone, which I called tester3, since my first couple attempts didn’t go so hot
  3. Install some packages
  4. Start some services
  5. Install some perl packages
  6. Give it a go!

After 1 and 2, let’s see where we are.  Just for kicks, let’s give our one-liner a shot.  I get

-bash: perl: command not found

Turns out that the zones create in OpenSolaris are pretty dang bare, as per this article.  In fact, many programs, like less and telnet, that I expected to have there, weren’t.  Using this page, and this command

  • pkg search -lr telnet

helped quite a bit.  Looks like my zone starts with 54 packages.  Sun’s Web Stack project provides an amp-dev package, so let’s install that and a couple other things for 3.

  • pkg install sunstudioexpress SUNWperl584usr amp-dev

As per the article above, the packages get pulled down and all together those three packages look to be 585.31 megs, so it takes a minute or ten to get downloaded via my cable modem connection.

Once installed I try again and get

  • Can’t locate DBI.pm in @INC (@INC contains: /usr/perl5/5.8.4/lib/i86pc-solaris-64int /usr/perl5/5.8.4/lib /usr/perl5/site_perl/5.8.4/i86pc-solaris-64int /usr/perl5/site_perl/5.8.4 /usr/perl5/site_perl /usr/perl5/vendor_perl/5.8.4/i86pc-solaris-64int /usr/perl5/vendor_perl/5.8.4 /usr/perl5/vendor_perl .) at -e line 1.  BEGIN failed–compilation aborted at -e line 1.

DBD::mysql seems to install better with mysql up and running, so as per the Administering MySQL Server section of this page, we

  • svccfg import /var/svc/manifest/application/database/mysql.xml
  • svcadm enable application/database/mysql:version_50

The DBD::mysql tests want to run mysql_config, so we need to get that in the PATH

  • export PATH=$PATH:/usr/mysql/5.0/bin

Now we’re ready to install DBI and DBD::mysql.  The first time or two through I tried installing the SUNWpmdbi package, but it didn’t play well (perhaps because it is like a year old), so instead I do

  • perl -MCPAN -e shell
  • install Test::More DBI DBD::mysql

Perhaps a little too hard, but now, so, nice!

  • root@tester3:~# perl -e “use DBI;my \$dbh=DBI->connect(‘DBI:mysql:database=mysql’, ‘root’, ”);print ‘ping=’ . \$dbh->ping . qq{\n}”
  • ping=1

Time to try out mod_perl.

If any Sun or web stack folks are reading, including packages for DBD::mysql and a more modern DBI would be great 🙂  And maybe adding the mysql bin to the path?

Enjoy!

Earl

thoughts?

Wednesday, October 29th, 2008

"So, I've been thinking of trying a little lipstick."

“So, I’ve been thinking of trying a little lipstick.”

I smell bacon!

Friday, October 24th, 2008

For years I have been interested in parsing logs.  Did a fair amount of it in a past life.  I think pig changes pretty well everything.  Back in the day, I wrote some parsing in perl, tried to figure out how to group things, colate, += these logs into a database, etc.  Well, for me, pig makes the parsing pretty slick.

My pig scripts generally look like

REGISTER coolJarOne.jar

REGISTER secondJar.jar

Those jars can contain user defined functions (UDFs), for doing custom log parsing.

Then, let’s slurp in a log

raw = LOAD ‘myLog.txt’ USING org.apache.pig.piggybank.storage.apachelog.CombinedLogLoader() AS (remoteAddr, remoteLogname, user, dayTime, method, uri, proto, status, bytes, referer, userAgent);

So that one line will parse one of apache’s combinedLogs.  And yeah, I wrote it.  It isn’t commited in yet, but I hope to submit a patch tonight-ish.

Then, this little bit of pig latin pulls out all the refers from my log

refererRaw1 = FOREACH raw GENERATE MyDateExtractor(dayTime), host, remoteAddr, referer;
refererRaw2 = FILTER refererRaw1 BY com.loghelper.MyLength($3) > 1;
refererRaw3 = GROUP refererRaw2 BY ($0, $1, $2, $3);
refererRaw4 = FOREACH refererRaw3 GENERATE FLATTEN($0), COUNT($1);
STORE refererRaw4 INTO ‘log-referers.txt’ USING PigStorage();

so log-referers.txt is a tab delimited file, which is rather easy to parse and += into mysql.  The beauty is that you can supposedly do it all in parallel.  Haven’t done that yet, but hopefully soon.

A few exciting things have been happening for me.

  1. My pig patches (which I call bacon) have been committed in.  Check it out.
  2. I tied in my pig stuff to parse logs from good old holaservers.
  3. I tied into the Google Visualization API and am now generating some pretty killer graphs.
  4. Tonight I took a friend’s combined log, ran it through my system and folded it on in to the above graphs.

Course, all the pig and hadoop (the parallel processing stuff) work will hopefully be the basis of good old loghelper.

where is my van?

Friday, October 3rd, 2008

(originally posted September 16th, 2008)

So, after my awesome marketing, I actually got some few signups.  And sadly, though perhaps not too surprisingly, most all the signups are spammers.  Looking like rather dumb linker sites, and some auto-redirect sites.  I nearly linked to them, but there you go.  Think I will block with rather extreme prejudice.  By email, by ip, each site for the offending user, etc.

I used to fight against such things at freeservers.  Sounds like I need to write a script or two that can detect problems and block offending sites.  Not really what I want to be working on, but I sure don’t want to pay for folks to peddle their spammy sites.

Finding offensive javascript is one thing, but I am thinking that for offensive images maybe I will use amazon’s mechanical turk for such things.  In order to get my feet wet, I have the following HIT (Human Intelligence Tasks) currently live

In English, please type in your favorite clean joke. Jokes that aren’t clean will not be accepted. You are welcome to provide an explanation for why you like your joke so much.
For a nickle a joke, it has been fun to watch.  Think my favorite weird response has been
There was one litle man whit his van and wherever he goes he say i am a litlle man this is my van and he robs a Bank.After 20 states finally he come’s in Bosnia and say i am a litlle man where is my van. I am from Bosnia and Hercegovina and this joke is one of many about bosnian people I had to change the joke a litlle bit to write in english but it didn’t lose his charm. I like this joke because most people Bosnian people survived everything and they are clever and most of them thives.
And the joke I will most likely keep repeating?
Q: What is brown and sticky?
A: a stick
Enjoy!
Earl

head in the clouds

Friday, October 3rd, 2008

(originally posted September 16th, 2008)

Recently I went to an amazon web service thing up in salt lake.  Turns out the cloud (they say) is all the rage.  Folks doing pretty well everything in the cloud.  I recently moved the start of holaservers to google app engine.  Here is a before pic

Pretty well every non-S3 thing was a single point of failure.

And now

So, for a bit of work, I have a highly available front page.  And for some reason, it is free until I start getting like five million hits a month.  Yeah, not really bumping up to that.

At the aws thing, I learned that I could do all my holaservers stuff in their cloud.  Didn’t really think I could do ftp or kind of hard web serving via my own custom apache, but yeah, can totally do both.  Sounds like EC2 can pretty well just give me a virtual opensolaris box.  I am kind of tempted to start moving stuff on over.  Besides a pretty big aws learning curve, and quite a bit of work, I would have to start putting out $72 a month, which won’t really be happening till I make some money or become a startup finalist.  Think I find out on October 3rd.

Earl