Tuesday, August 05, 2008

AdSense Secret - How to choose the best keywords for adSense

Ever wondered how to select the best keywords for your Adsense websites? Your focus is earning money as well as attracting organic traffic towards your website. Here is what you need to know while choosing a keyword, a scientific study that maps the ideal keywords for your next upcoming adsense website.

We shall first study how to choose keywords that lead to more Adsense money.

CPC - Advantages and Drawbacks

What does CPC indicate?

When we think of high paying keywords, the first thing that pops up in our mind is CPC. CPC (Cost Per Click) is the maximum amount of money an advertiser is willing to pay for a click. Higher the CPC of a particular keyword, more will be the payouts you can expect by targeting that keyword in your AdSense websites. You can get the most accurate CPC from the source itself - Google.

However, CPC is not the only factor that tells the profitability of a keyword. There are many other factors that you should consider.

Drawbacks of CPC

- What if there are no advertisers bidding on a keyword having high CPC? Obviously, it means no one is going to pay you that high for a click.
- What if the traffic on the keyword is technical enough and does not click on ads (banner blindness)?
- What if Google gets loads of traffic on that keyword, enough to fulfill the desires of high paying advertisers? In this case, Google will only throw remaining peanuts towards you.

Obviously, Google will like to keep all the high paying ads within its own pages. So you see, CPC just shows you a small part of the entire picture. We shall explore the rest part of the picture...

Number of Sponsors - The Most Important Figure

Keywords with high CPC do not always come with high number of advertisers. There are many keywords in Google's Adwords system that have a high price, but almost no or very less number of advertisers. Now, if you build a web page or website around a keyword on which no one is bidding, Google will try to fill the ad blocks with ads of the related keywords rather than the high paying keyword you were initially targeting. There is no guarantee that ads of the related keywords will be good converters. Therefore, there is every likelihood that your clicks will be passed through SmartPricing Filter, eventually resulting in lesser payouts.

Hence, it is important to see the number of sponsors bidding on the keyword in question. If there are more the advertisers, there will be more competition among them to get the top position. This means that there will be more revenue per click that they are willing to share with you. In other words, more sponsors mean more people fighting against one another other to pay you more Adsense money.

Number of clicks on an Ad

It refers to the number of clicks an advertiser gets when his ad appears on the top. If the number of clicks and number of advertisers is less, just ignore those keywords, no matter what their CPC is. However, if the clicks per month are lower than expected, but the number of advertisers is a good one - it's the right choice for you. Such keywords are called "Niche AdSense' keywords." Here the word 'niche' is used in commercial context. This means that Google is not able to generate traffic on such keyword ads. This makes it difficult for it to exhaust the advertising budgets of the advertisers. Therefore, in order to meet its targets, Google happily shares high paying ads with you, resulting in more payouts. Try to find such "Niche AdSense' keywords."

Coming to Higher number of clicks - It indicates:

Either the traffic segment is ignorant about the online advertising concepts, and click on these ads unknowingly. This means you will experience higher CTR on your ads when placed at appropriate place. You don't have to put in much efforts to fight banner blindness.

Or, the traffic is highly commercial and willing to purchase the advertised product over internet. This means that there are very little chances of your facing the SmartPricing phenomenon.

Number of clicks along with the other stats like Number of sponsors and CPC, can enable you to make more wise decisions while choosing keywords for your AdSense content.

Bidding Quality

It's important to see the pattern in which people are bidding on a keyword. Suppose there are 400 advertisers bidding on a keyword. The top 20 of these are paying something like $15 per click, while the rest of them pay somewhere between $2 and $0.05. Now you might have found some decent tools that give you the average of top 3 or top 5 positions which is good, but not good enough. The point is that although the top 20 advertisers are paying higher, but the rest 380 advertisers are paying quite low. There is a high probability of your getting the ads of those 380 sponsors. Therefore, the average of the top 20 advertises can be really misleading. The solution to this problem is discussed in the later part of this page.

We have enough Adsense money now. Let's build some traffic on your website.

Traffic Building for AdSense:

Choosing Niche Keywords

Niche keywords are the keywords that are highly searched by the web surfers, and are rarely used by your competitors. Less competition means more traffic to your website. Targeting ten niche keywords is easier and more fruitful that targeting a highly competitive keyword. Traffic from niche keywords when directed to a relevant page increases your CTR and conversion ratio.

Determining Competition

People generally take the number of results returned by search engines as the number of pages competing on a keyword. But it is wrong. The Search Engine Results get irrelevant after 10 - 15 pages. Irrelevancy further increases with the depth. The pages that have the keyword dumped in a corner are not competing against you, but search engines will still list them. In fact they have to.

It is assumed that if a webmaster is targeting a web page with a particular keyword, the keyword is used in the title as well as in the anchor text linking to that webpage. Such a page is listed higher by the search engines as it is dedicated to what you searched for. So how to filter out the most relevant results? Check it out!

Inanchor intitle and Its Precision - The Solution

In Google, you can easily determine the EXACT number of pages that are competing against you. You can precisely list out the pages that are using a particular keyword in their page titles or in the anchor texts linking to them.

The query can be applied as follows: intitle:keyword inanchor:keyword.

For example, if the keyword is "hair treatment", the formula will be used in the following manner: intitle:hair inanchor: hair intitle: treatment inanchor: treatment. This figure gives you the exact number of pages that are ACTUALLY Targeting with these keywords, and not those that have just created a page or a small paragraph on the same topic. Google emphasizes on Anchors and Page titles. That's the reason, it supports such a search query.

We shall now discuss some other traditional ways to determine competition.

R/S Ratio

Here, R refers the number of competitor websites for a particular keyword as per the search result of the search engine. And S refers the number of searchers using that keyword while searching their queries. This means that for better results, you have to choose the keywords with lower R/S ratio.

R/S ratio becomes polluted when someone uses the number of results as the number of competing websites. As explained above, counting the number of results as the number of competing pages is the biggest mistake one can make while choosing a keyword. However, the figure becomes quite useful when inanchor intitle is used to create R/S.

KEI Analysis

KEI (Keyword Effectiveness Index) is a formula for measuring the effectiveness of a keyword. The formula was devised by Sumantra Roy. However, this figure also depends upon the number of searches and competition, but with a difference. This formula analyzes the number of searches and competition in such a way that if the searches increase, KEI increases; and if the competition increases, KEI decreases. Higher the KEI, more profitable will be the keyword. However, it becomes polluted when the number of search results are used as the number of competitors.

Determining Traffic

Determining traffic for a keyword is quite important before targeting it. Along with the competition stats, it lets you make out the niches present in any industry. Besides, it lets you predict (to some extent) how much traffic you can expect if you promote a website around a particular keyword. There are two known sources for determining traffic. Overture Keyword Suggestion Tool (now a part of Yahoo Search Engine) and WordTracker. When talking about accuracy, Wordtracker monitors the queries on some meta search engines that actually render it inaccurate . Reason? There is no known stat about how much of the community you are targeting is searching on those meta search engines. However, it's a good tool to make out the niches.

In my opinion, Yahoo owned Overture gives you more accurate stats than Wordtracker. This is for the reason that Overture is a PPC engine. It can show you how many people are searching for a particular term on its vast network. A network which is bigger than any Meta Search Engine. Still, it is not that accurate, but at least better than Wordtracker. The tools that are predicting Google Searches for you are just doing guess work. No one knows the algorithm they use.

Summing It Up

Profitability of a keyword depends more on the number of sponsors than CPC. Number of clicks and bidding quality can't be determined by just monitoring the first 3 or 8 places. It can be rightly assessed by taking out the average of all the 400 sponsors bidding on a keyword. Sounds very tiring for keyword research, right?

Wednesday, July 23, 2008

20 ways to Secure your Apache Configuration

Here are 20 things you can do to make your apache configuration more secure.

Disclaimer: The thing about security is that there are no guarantees or absolutes. These suggestions should make your server a bit tighter, but don't think your server is necessarily secure after following these suggestions.

Additionally some of these suggestions may decrease performance, or cause problems due to your environment. It is up to you to determine if any of the changes I suggest are not compatible with your requirements. In other words proceed at your own risk.

First, make sure you've installed latest security patches

There is no sense in putting locks on the windows, if your door is wide open. As such, if you're not patched up there isn't really much point in continuing any longer on this list. Go ahead and bookmark this page so you can come back later, and patch your server.

Hide the Apache Version number, and other sensitive information.

By default many Apache installations tell the world what version of Apache you're running, what operating system/version you're running, and even what Apache Modules are installed on the server. Attackers can use this information to their advantage when performing an attack. It also sends the message that you have left most defaults alone.

There are two directives that you need to add, or edit in your httpd.conf file:

ServerSignature Off
ServerTokens Prod

The ServerSignature appears on the bottom of pages generated by apache such as 404 pages, directory listings, etc.

The ServerTokens directive is used to determine what Apache will put in the Server HTTP response header. By setting it to Prod it sets the HTTP response header as follows:

Server: Apache

If you're super paranoid you could change this to something other than "Apache" by editing the source code, or by using mod_security (see below).

Make sure apache is running under its own user account and group

Several apache installations have it run as the user nobody. So suppose both Apache, and your mail server were running as nobody an attack through Apache may allow the mail server to also be compromised, and vise versa.

User apache
Group apache

Ensure that files outside the web root are not served

We don't want apache to be able to access any files out side of its web root. So assuming all your web sites are placed under one directory (we will call this /web), you would set it up as follows:


Order Deny,Allow
Deny from all
Options None
AllowOverride None


Order Allow,Deny
Allow from all

Note that because we set Options None and AllowOverride None this will turn off all options and overrides for the server. You now have to add them explicitly for each directory that requires an Option or Override.

Turn off directory browsing

You can do this with an Options directive inside a Directory tag. Set Options to either None or -Indexes

Options -Indexes

Turn off server side includes

This is also done with the Options directive inside a Directory tag. Set Options to either None or -Includes

Options -Includes

Turn off CGI execution

If you're not using CGI turn it off with the Options directive inside a Directory tag. Set Options to either None or -ExecCGI

Options -ExecCGI

Don't allow apache to follow symbolic links

This can again can be done using the Options directive inside a Directory tag. Set Options to either None or -FollowSymLinks

Options -FollowSymLinks

Turning off multiple Options

If you want to turn off all Options simply use:

Options None

If you only want to turn off some separate each option with a space in your Options directive:

Options -ExecCGI -FollowSymLinks -Indexes

Turn off support for .htaccess files

This is done in a Directory tag but with the AllowOverride directive. Set it to None.

AllowOverride None

If you require Overrides ensure that they cannot be downloaded, and/or change the name to something other than .htaccess. For example we could change it to .httpdoverride, and block all files that start with .ht from being downloaded as follows:

AccessFileName .httpdoverride

Order allow,deny
Deny from all
Satisfy All

Run mod_security

mod_security is a super handy Apache module written by Ivan Ristic, the author of Apache Security from O'Reilly press.

You can do the following with mod_security:

  • Simple filtering
  • Regular Expression based filtering
  • URL Encoding Validation
  • Unicode Encoding Validation
  • Auditing
  • Null byte attack prevention
  • Upload memory limits
  • Server identity masking
  • Built in Chroot support
  • And more

Disable any unnecessary modules

Apache typically comes with several modules installed. Go through the apache module documentation and learn what each module you have enabled actually does. Many times you will find that you don't need to have the said module enabled.

Look for lines in your httpd.conf that contain LoadModule. To disable the module you can typically just add a # at the beginning of the line. To search for modules run:

grep LoadModule httpd.conf

Here are some modules that are typically enabled but often not needed: mod_imap, mod_include, mod_info, mod_userdir, mod_status, mod_cgi, mod_autoindex.

Make sure only root has read access to apache's config and binaries

This can be done assuming your apache installation is located at /usr/local/apache as follows:

chown -R root:root /usr/local/apache
chmod -R o-rwx /usr/local/apache

Lower the Timeout value

By default the Timeout directive is set to 300 seconds. You can decrease help mitigate the potential effects of a denial of service attack.

Timeout 45

Limiting large requests

Apache has several directives that allow you to limit the size of a request, this can also be useful for mitigating the effects of a denial of service attack.

A good place to start is the LimitRequestBody directive. This directive is set to unlimited by default. If you are allowing file uploads of no larger than 1MB, you could set this setting to something like:

LimitRequestBody 1048576

If you're not allowing file uploads you can set it even smaller.

Some other directives to look at are LimitRequestFields, LimitRequestFieldSize and LimitRequestLine. These directives are set to a reasonable defaults for most servers, but you may want to tweak them to best fit your needs. See the documentation for more info.

Limiting the size of an XML Body

If you're running mod_dav (typically used with subversion) then you may want to limit the max size of an XML request body. The LimitXMLRequestBody directive is only available on Apache 2, and its default value is 1 million bytes (approx 1mb). Many tutorials will have you set this value to 0 which means files of any size may be uploaded, which may be necessary if you're using WebDAV to upload large files, but if you're simply using it for source control, you can probably get away with setting an upper bound, such as 10mb:

LimitXMLRequestBody 10485760

Limiting Concurrency

Apache has several configuration settings that can be used to adjust handling of concurrent requests. The MaxClients is the maximum number of child processes that will be created to serve requests. This may be set too high if your server doesn't have enough memory to handle a large number of concurrent requests.

Other directives such as MaxSpareServers, MaxRequestsPerChild, and on Apache2 ThreadsPerChild, ServerLimit, and MaxSpareThreads are important to adjust to match your operating system, and hardware.

Restricting Access by IP

If you have a resource that should only by accessed by a certain network, or IP address you can enforce this in your apache configuration. For instance if you want to restrict access to your intranet to allow only the 176.16 network:


Order Deny,Allow
Deny from all
Allow from 176.16.0.0/16

Or by IP:

Order Deny,Allow
Deny from all
Allow from 127.0.0.1

Adjusting KeepAlive settings

According to the Apache documentation using HTTP Keep Alive's can improve client performance by as much as 50%, so be careful before changing these settings, you will be trading performance for a slight denial of service mitigation.

KeepAlive's are turned on by default and you should leave them on, but you may consider changing the MaxKeepAliveRequests which defaults to 100, and the KeepAliveTimeout which defaults to 15. Analyze your log files to determine the appropriate values.

Run Apache in a Chroot environment

chroot allows you to run a program in its own isolated jail. This prevents a break in on one service from being able to effect anything else on the server.

It can be fairly tricky to set this up using chroot due to library dependencies. I mentioned above that the mod_security module has built in chroot support. It makes the process as simple as adding a mod_security directive to your configuration:

SecChrootDir /chroot/apache

There are however some caveats however, so check out the docs for more info.

Acknowledgments

I have found the book Apache Security to be a highly valuable resource for securing an apache web server. Some of the suggestions listed above were inspired by this book.

Suggestions

Please post any suggestions, caveats, or corrections in the comments and I will update the post if necessary.

Tuesday, July 22, 2008

Some basic Unix/Linux network commands for Troubleshoot

Troubleshooting or just need a quick refresher on some basic and advanced Unix Linux network related commands?

ifconfig - configure a network interface (setup)
route - show / manipulate the IP routing table
ping, ping6 - send ICMP ECHO_REQUEST to network hosts
netstat
- Print network connections, routing tables, interface statistics, masquerade connections, and multicast memberships

tcpdump - dump traffic on a network
tcpslice - extract pieces of and/or glue together tcpdump files

traceroute - print the route packets trace to network host
tracepath, tracepath6 - traces path to a network host discovering MTU along this path
iwconfig - configure a wireless network interface
iwlist - Get more detailed wireless information from a wireless interface
wpa_supplicant - Wi-Fi Protected Access client and IEEE 802.1X supplicant
wpa_supplicant.conf [wpa_supplicant] - configuration file for wpa_supplicant
ipcalc - perform simple manipulation of IP addresses
nc - arbitrary TCP and UDP connections and listens
snort - open source network intrusion detection system
ettercap - A multipurpose sniffer/content filter for man in the middle attacks
ethereal - Interactively dump and analyze network traffic