: the classification of goods repeat
page classification of goods commodity list if there is more than one, it will produce the same page, the spider will crawl, crawl page address, and ECSHOP default classification paging address is different from that of URL, but the title is the same description. This will lead to the same content, Shanghai dragon is very unfavorable. > and
simple modification way: in the root directory of the web site feed.php found the following code: $link = $uri. $item_url. $separator.’from=rss’; it replaced $link = $uri. $item_url; and then refresh the page feed.php to see whether there is no
in a web log analysis in the Shanghai, love a spider crawling with multiple parameters of dynamic web site, and then copy and paste directly open the URL in the browser, even the content appears to be one of the classification of goods. Later to find information, to know those parameters is an address by lifting order time, or the price of the renewed popularity. This is the Shanghai dragon website optimization is not friendly, is tantamount to duplicate content, can be seen from the chart, the author has been included. Solution: the suggestion can use nofollow do not track the dynamic link when it launched a new station, with a plurality of dynamic parameter address. Some might say that the program can not get rid of? If it is removed, the user experience is not good, so to both.
address of goods;
for reading the source address should be and the original web address is the same, but I saw in the analysis of a log, the spider crawling first love Shanghai updated reading source, and with & from=rss, search for information, the original is ECSHOP in the feed.php file is added between? From=rss, love of spiders in Shanghai grab included, there will be a & from=rss page, the list of goods; to repeat capture, it is best to get rid of.
two: read the source feed.php program with & from=rss
three: page classification of goods list page with a plurality of dynamic parameters and different URL same title
because I used ECSHOP to build a fashion mall type site, every day of the web log analysis. The spider crawling traces summarizes ECSHOP needs but neglected three optimization details, absolutely let you share the content of dry cargo, nonsense not say, look at the following three points: