By Flexo

If your RSS feed is being “scraped” and republished in full somewhere else — as is currently happening to my main blog, Consumerism Commentary — you run two important risks. First of all, your scraper is likely pasting ads alongside your content on his website, trying to earn income directly off your hard efforts. Secondly, there’s a possibility that you can be penalized by the search engines for duplicate content, so you indirectly lose income.

This is a tough problem to combat, especially if you use a service like FeedBurner. I use FeedBurner for my main blog’s RSS feed because of the many interesting statistic-related features available. I’ve never published my FeedBurner address as my main feed. Rather, I used redirects, so those subscribed to my original feed at consumerismcommentary.com/index.xml never had broken links. That is the only URL I’ve ever published for the URL, so it is the one that was discovered by scrapers.

If I had publicized my feed, FeedBurner address, the scrapers would be using that, and I would not be able to do what I am doing today to avoid being scraped.

Now, for dealing with the scrapers.

Step 1. Create a fake feed. There’s no need to make this difficult. Take Consumerism Commentary’s fake feed, edit the links and text to your liking, and save or upload as fakefeed.xml on your web server. The item dates are in the future. This future-dating could have been done with simple PHP to automatically advance the dates and keep the feed fresh, but I was a little lazy.

Step 2. Locate your scrapers. The software that scrapes RSS feeds resides on the same machine as the website publishing the stolen content. Simply use nslookup in DOS or UNIX/Linux to determine the IP address of that machine. Let’s say the IP of the first offending machine is 128.175.13.63. Let’s also say you have a second offending machine at 192.193.217.120.

Step 3. Redirect requests to the fake feed. To redirect http requests, you’ll need access to your .htaccess file in your domain’s mapped root folder. Add the following to your .htaccess file:

RewriteEngine On
RewriteBase /

RewriteRule ^fakefeed.xml - [S=300]

RewriteCond %{REMOTE_ADDR} ^128\\.175\\.13\\.63$ [OR]
RewriteCond %{REMOTE_ADDR} ^192\\.193\\.217\\.120$
RewriteRule .* http://www.consumerismcommentary.com/fakefeed.xml [R=302,L]

The first two lines may already exist. If they do, there’s no need to duplicate. Just place the rest of the above content below the already existing lines. Of course, replace the URL in the second RewriteRule with the URL to your fake feed.

The [S=300] in line 3 advises the web server to skip the next 300 commands in the .htaccess file if fakefeed.xml is requested. This is important, because otherwise your server will enter an endless loop of redirects. In the highly unlikely event that you have an incredibly long .htaccess file, you may have to increase the number from 300 to another choice.

That’s it! Once the scrapers realize you’re providing bad content, they’ll likely stop scraping your feed. If they don’t stop, at least they’re not taking your real content’s full entries or excerpts.

Note: It’s too late to do anything about the entries that have already been scraped. Depending on the scraper’s software, these may not go away until the perpetrator removes the entire website. This will prevent future items from being scraped… until a new IP address is used.

By nickel

Here’s a bit of WordPress hackery for you… In case you haven’t noticed, WordPress 2.1 has changed the way posts and Pages are defined in the database. What this means to you that a variety of plugins for generating archives, lists of related posts, etc. started including Pages along with posts following the upgrade. Not devastating, but kind of annoying. So here’s the fix… Look at the plugin code for instances of:

post_status=’publish’

and change it to:

post_status = ‘publish’ AND post_type = ‘post’

I’ve tried this on two different plugins (Smart Archives and WASABI Related Entries) and it worked in both cases. Only later did I realize that Justin Blanton had already released a newer version of Smart Archives that fixed the problem. Oh well, at least I learned something.

By nickel

Just a quick note for you Firefox users out there… In case you haven’t heard, there’s a great little in-browser weblog editor called ScribeFire that’s free for the taking. ScribeFire actually started out as a Firefox extension from the good folks at Performancing but was later spun off and re-branded. You really have to try it out to fully appreciate it, but here’s a synopsis…

It hangs out as a little orange pen and paper icon down in the lower right corner of your Firefox window. When you click on it, however, it creates a split-screen with web pages being displayed up top and a resizeable ScribeFire pane in the bottom. The web browser pane is fully functional, allowing you to surf around, manage tabs, etc., all while having access to the editor pane. Moreover, you can ScribeFire it to manage multiple weblogs, and it supports categories, trackbacks, and Technorati tags such that you can publish straightaway from within ScribeFire. You can also pull up previous posts and edit them. You can also save what you’re working on locally as a ‘note’ or publish it to your site as a draft. The editor works either as an old-school source editor or as a WYSIWYG editor.

While I’ve only used it with WordPress, and thus can’t vouch for it’s utility with other platforms, I highly recommend that you give it a try. It has the potential to revolutionize your workflow! There’s nothing better than being able to view other web sites while simultaneously writing! No need to flip back and forth between tabs, etc. You can download it here.

While we’re on the subject, I’d love to hear about any indispensable blogging tools that you’ve run across.

By jdroth

I have a friend who is starting a niche personal finance blog. He’s very interested in the subject, and knowledgeable, and I think he could make it an interesting site.

When the idea first came to him a few months ago, he approached me. “How do I make this a successful blog?” he asked.

“Post lots of good content,” I told him.

“Yeah,” he said. “But what else?”

“There is no ‘what else’,” I said. Actually, I ranted and raved about how too many people focus on things that aren’t important and don’t bother to spend time on the content, but essentially it all amounted to “there is no ‘what else’”.

My friend went away for a few weeks to work on his site. When I talked to him again he told me, “I’ve switched from WordPress to Drupal. Do you think that’ll make a difference?”

“It’ll make no difference at all,” I said. “Readers don’t care what weblog tool you use. All they care about is the content.”

“Yeah, but Drupal offers so many more features,” he said. I just shook my head.

About a month ago, he launched his site. He posted an introductory article. “Looks good,” I said.

“Can you point people to the site?” he asked.

“Not yet,” I said. “You don’t have any content.”

Meanwhile he put up some Google ads and some Amazon ads. He posted a single link to another article at a big news site. I talked to him a couple weeks later. “Nobody’s coming to my site,” he told me. “Not a single person has clicked on an ad.”

“That’s because there’s nothing there,” I told him.

“What do you mean?” he said. “I spent a lot of time creating the layout and putting up the ads.”

“You need to focus on content,” I told him.

So he wrote another article. It was moderately interesting, but it was all in one h-u-g-e paragraph. That was on February 21st. There’s been nothing new posted to the site since then. The site layout has changed a half-dozen times, though, as my friend looks to make it as pretty as possible.

He IMed me last night. “Nobody’s coming to the site,” he said.

“It needs content,” I told him.

“I don’t have time,” he said. “I’m so busy.” I pointed out that he wasn’t too busy to party with friends. He wasn’t too busy to play soccer. He wasn’t too busy to tinker with the layout. These are all fine things, but none of them have anything to do with getting readers. “Can’t you point people to my site?” he asked.

“Maybe in a couple months,” I said. “Maybe once you have some content.”

This concept has been beat into the ground a thousand times before, but it’s the single most important factor in creating a successful weblog: To gain readers, you must publish quality content on a regular basis. Sure, readers like a pretty site. Sure, it would be nice if there were ads for them to click on. But all of this is secondary. All that really matters is the content.

That, my friends, is all that you ever need to know about blogging.

By nickel

“If you have an apple and I have an apple and we exchange these apples, then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas.”
–George Bernard Shaw

By jdroth

Amy at The Writer’s Group declares, “Enough!” She gives a few anecdotes of writers who have achieved some measure of success, but for whom this is not enough.

Lately I’ve fallen victim to creeping expectations. Others’ expectations, yes, but more than that, my own expectations. I let past success intimidate me so that I’m afraid to continue trying. This is dangerous ground.

Amy concludes by writing:

Relax. Write the very best you can with each book. Take your time and avoid anyone else’s expectations. Have goals, but make sure they’re your own. Stay true to your story and to your self. Write for the pure joy it gives you. Let that be enough — at least for today.

Replace “blog” for “book” and all I can say is: Amen.

By nickel

Over the past couple of weeks, I’ve been experimenting with different methods for cutting down on comment spam on both FiveCentNickel and Raising4Boys. This all started when I upgraded to WordPress 2.1. In the past, I’ve used SpamKarma 2 and had great luck with it. But as it turns out, things have been rearranged a bit in WP 2.1, and the comment management links (e.g., mark as spam, delete comment, etc.) that come in the SK2 comment notification e-mails no longer point where they’re supposed to. Thus, I switched to Akismet.

Unfortunately, Akismet has a much higher rate of false-positives on my sites than does SK2. This wouldn’t be a huge deal if I didn’t get so much spam, but it’s virtually impossible to wade through my spam folder looking for legit comments that got accidentally filtered out. After a couple of days of struggling with this problem, I decided to be proactive. What follows is a rundown of the various things that I’ve tried, culminating with the winning combination…

First off, I installed the Akismet Worst Offenders plugin, which sorts through your spam, aggregates the common offenses into bunches, and then allows you to delete entire classes of spam all at once. This makes it dead easy to throw out the vast majority of your spam without actually looking at it. You can then sort through the leftovers looking for false-positives. The other nice thing about this plugin is that it automatically bans the offending IP addresses via .htaccess when you delete the associated spams. This is a great tool for spam management, but it doesn’t do that much to reduce your spam load, despite the IP blocking.

Second, I tried a cool .htaccess trick that I found in the WordPress Codex. In short, this allows you to deny spambots access to your comment form if they don’t have a valid referrer from somewhere within your site. The underlying theory here is that, because an most spambots hit wp-comments-post.php directly without following a link from one of your posts (as a valid commenter would do), you can block them automagically without interfering with legitimate visitors.

At first, this seemed like a dream solution. Spam slowed to a trickle, and all was good. Then I got an e-mail from a regular commenter saying that he couldn’t leave comments. It turns out that he was using a proxy server, and it wasn’t reporting a valid referrer. Since blocking loyal readers isn’t an option for me, I had to drop this one.

Next up: Installing a small check to test whether or not a commenter is human. I’m not a of graphic CAPTCHAs, so I went with a math plugin instead. While some spam bots seem to parse the math problem and still get through, this works pretty well, and my spam levels fell noticeably. A side benefit of this plugin is that it keeps profoundly dumb people from commenting. ;)

The final step that I took was to change the name of wp-comments-post.php to something else (thanks to Jim for the suggestion). As I noted above, many spambots hit this file directly, so if you rename it, they won’t be able to find it. This works great! The only trick here is to make sure you change any calls to this file from within your theme, otherwise your comments won’t work. Fortunately, this involved opening up comments.php and changing it in just one spot. Of course, since you’re renaming a core piece of the WordPress code, you’ll need to re-do this when you upgrade to the next version.

So that’s where things currently stand… I’ve renamed wp-comments-post.php, installed the math plugin, and I’m filtering with Akismet (helped along by the Worst Offenders plugin).

While this combination has reduced spam to about 10% of the original level, a bit still gets through. Thus, I’ve modified the text that surrounds the math problem to make the nature of the underlying test slightly less obvious (it’s a bit less “mathy” and a bit more “wordy”). Hopefully this will make the math problem more difficult to parse, and will push the spam levels even lower.

So there you have it… If you have any spam-defeating tricks up your sleeve, please share them here in the comments.

Note: Before anyone suggests Bad Behavior, I’ll just say that I’ve tried it, and it worked great. Unfortunately, it also blocked a small number of legitimate visitors and, as I noted above, that’s unacceptable to me.

By jdroth

It was late last year that I realized I could potentially make a living writing for the web. It was today that I knew that this was true. I make a modest (but decent) income at the box factory. But for the last week, my web income has equaled my income from my real job. Scary, huh?

Now this is just one week. Though I’m making good money from my writing, there are many ups and downs. But even the lows are higher than I could have imagined. On November 25th, I made $29.29 in web income. That is the last day my earnings dipped below $30. My best day was last Tuesday: I made $169.90.

Over at 2blowhards (still one of my favorite blogs), Michael writes:

Planning on getting rich writing sci-fi or fantasy novels? Think again. Tobias Buckell writes that the average advance for a first sci-fi or fantasy novel is $5000. Five years and five novels later, the average author is pulling in around $13,000 per novel.

I used to want to get rich off writing sci-fi or fantasy. Then I decided I just wanted to get rich off writing books — I didn’t care what kind. More and more, it’s clear that I may never publish a book (at least not in the traditional sense)! I’m already making twice what a sci-fi novelist makes, and I have complete control of my content. There’s little motivation for me to change directions at the moment.

Some people — and perhaps you’re one of them — look disdainfully upon web income. “You’re not making money from writing,” is a common observation. “You’re making money from advertising.” I can understand this delineation, but it’s not one that I make.

I am writing, and publishing that writing, and it’s making me money. I don’t feel guilty about it. I don’t feel as if I’m compromising anything. Did I ever dream I’d make a living writing about personal finance? Nope. But now I can’t imagine anything else I’d rather be doing.

By nickel

After thinking long and hard about this, I decided to remove some of the Google Adsense blocks from FiveCentNickel and Raising4Boys about ten days ago. All of the positions were performing reasonably well, but I wanted to unclutter things a bit. Thus, I decided to tackle the weakest performing slots. I removed the right sidebar skyscraper from throughout the site, and also removed the ad block from the center column on the front page (between the first and second entry).

It felt great to free up that extra space, and I quickly took advantage of it by adding more widgets to the right sidebar. The next day, however, I started questioning the wisdom of this move. Adsense performance was down. Way down — by about 50%. Then again, sometimes you just have a bad day. Since I was already grooving on the lower density of ads, I decided to just sit tight. And then something interesting happened… My earnings rebounded. Big time. Over the next couple of days, Adsense performance returned to normal, and as of this writing, I’m actually doing better than I was before I removed the Adsense.

I still haven’t had time to sort out everything that’s gone on, but my overall clickthrough rate (CTR) is about what it was before I removed the extra ads, and the cost per click (CPC) is likewise unchanged. Thus, it appears that the Adsense activity from the now defunct ad slots has been re-focused on the remaining ad slots. Whatever the cause, I couldn’t be happier with the outcome. It seems that I can have my cake and eat it, too!

By Flexo

I received an invitation from Google to try out AdSense Video Distribution, a new advertising delivery format. The video content is provided by The Wall Street Journal in my case, and advertisements are placed between clips. Other content providers are available.

Google is providing a wide selection to match various types of publishers, which I assume will expand as the program extends beyond this initial “pilot” period.

For this pilot we have selected content in the music, entertainment, finance, lifestyle, and sports categories. We’re working with content providers such as The Wall Street Journal Online, LX.TV, Warner Music Group, Sony BMG Music Entertainment, and CondeNet’s Epicurious and Style.com.

Revenue is generated (and split between the publisher, Google, and the ad content provider) only when a visitor begins watching the advertisement videos between clips. The visitor also has the option to skip past any part of the video.

It sounds interesting, and I’d be interested to see how it compares with AdSense’s text ads. Take a look at how the ad is incorporated at Consumerism Commentary, where I am using it only on the main index. My current layout isn’t very conducive to placing this type of ad above the fold, and I don’t want to be accused of over-monetizng. I pushed several other AdSense ads — the video ads don’t count towards the three units per page limit — further down.

I was hoping to hire a designer to redesign Consumerism Commentary, and make ads ess intrusive, but that’s not working out at the moment.