Crawling the Web
Gautam Pant1,Padmini Srinivasan1,2,and Filippo Menczer3
1Department of Management Sciences
2School of Library and Information Science
The University of Iowa,Iowa City IA52242,USA
email:gautam-pant,padmini-srinivasan@https://www.doczj.com/doc/585332644.html,
3School of Informatics
Indiana University,Bloomington,IN47408,USA
email:?l@https://www.doczj.com/doc/585332644.html,
Summary.The large size and the dynamic nature of the Web highlight the need for continuous support and updating of Web based information retrieval systems. Crawlers facilitate the process by following the hyperlinks in Web pages to automat-ically download a partial snapshot of the Web.While some systems rely on crawlers that exhaustively crawl the Web,others incorporate“focus”within their crawlers to harvest application or topic speci?c collections.We discuss the basic issues related with developing a crawling infrastructure.This is followed by a review of several topical crawling algorithms,and evaluation metrics that may be used to judge their performance.While many innovative applications of Web crawling are still being invented,we take a brief look at some developed in the past.
1Introduction
Web crawlers are programs that exploit the graph structure of the Web to move from page to page.In their infancy such programs were also called wanderers,robots,spiders,?sh,and worms,words that are quite evocative of Web imagery.It may be observed that the noun‘crawler’is not indicative of the speed of these programs,as they can be considerably fast.In our own experience,we have been able to crawl up to tens of thousands of pages within a few minutes.4
From the beginning,a key motivation for designing Web crawlers has been to retrieve Web pages and add them or their representations to a local repos-itory.Such a repository may then serve particular application needs such as those of a Web search engine.In its simplest form a crawler starts from a seed page and then uses the external links within it to attend to other pages. 4on a Pentium4workstation with an Internet2connection.
2G.Pant,P.Srinivasan,F.Menczer
The process repeats with the new pages o?ering more external links to follow, until a su?cient number of pages are identi?ed or some higher level objective is reached.Behind this simple description lies a host of issues related to net-work connections,spider traps,canonicalizing URLs,parsing HTML pages, and the ethics of dealing with remote Web servers.In fact a current generation Web crawler can be one of the most sophisticated yet fragile parts[5]of the application in which it is embedded.
Were the Web a static collection of pages we would have little long term use for crawling.Once all the pages had been fetched to a repository(like a search engine’s database),there would be no further need for crawling.However,the Web is a dynamic entity with subspaces evolving at di?ering and often rapid rates.Hence there is a continual need for crawlers to help applications stay current as new pages are added and old ones are deleted,moved or modi?ed.
General purpose search engines serving as entry points to Web pages strive for coverage that is as broad as possible.They use Web crawlers to maintain their index databases[3]amortizing the cost of crawling and indexing over the millions of queries received by them.These crawlers are blind and exhaustive in their approach,with comprehensiveness as their major goal.In contrast, crawlers can be selective about the pages they fetch and are then referred to as preferential or heuristic-based crawlers[10,6].These may be used for building focused repositories,automating resource discovery,and facilitating software agents.There is a vast literature on preferential crawling applica-tions,including[15,9,31,20,26,3].Preferential crawlers built to retrieve pages within a certain topic are called topical or focused crawlers.Synergism between search engines and topical crawlers is certainly possible with the lat-ter taking on the specialized responsibility of identifying subspaces relevant to particular communities of users.Techniques for preferential crawling that focus on improving the“freshness”of a search engine have also been suggested [3].
Although a signi?cant portion of this chapter is devoted to description of crawlers in general,the overall slant,particularly in the latter sections,is towards topical crawlers.There are several dimensions about topical crawlers that make them an exciting object of study.One key question that has moti-vated much research is:How is crawler selectivity to be achieved?Rich contex-tual aspects such as the goals of the parent application,lexical signals within the Web pages and also features of the graph built from pages already seen —these are all reasonable kinds of evidence to exploit.Additionally,crawlers can and often do di?er in their mechanisms for using the evidence available to them.
A second major aspect that is important to consider when studying crawlers,especially topical crawlers,is the nature of the crawl task.Crawl characteristics such as queries and/or keywords provided as input criteria to the crawler,user-pro?les,and desired properties of the pages to be fetched (similar pages,popular pages,authoritative pages etc.)can lead to signi?-cant di?erences in crawler design and implementation.The task could be con-
Crawling the Web3 strained by parameters like the maximum number of pages to be fetched(long crawls vs.short crawls)or the available memory.Hence,a crawling task can be viewed as a constrained multi-objective search problem.However,the wide variety of objective functions,coupled with the lack of appropriate knowledge about the search space,make the problem a hard one.Furthermore,a crawler may have to deal with optimization issues such as local vs.global optima[28].
The last key dimension is regarding crawler evaluation strategies neces-sary to make comparisons and determine circumstances under which one or the other crawlers work https://www.doczj.com/doc/585332644.html,parisons must be fair and made with an eye towards drawing out statistically signi?cant di?erences.Not only does this require a su?cient number of crawl runs but also sound methodologies that consider the temporal nature of crawler outputs.Signi?cant challenges in evaluation include the general unavailability of relevant sets for particular topics or queries.Thus evaluation typically relies on de?ning measures for estimating page importance.
The?rst part of this chapter presents a crawling infrastructure and within this describes the basic concepts in Web crawling.Following this,we review a number of crawling algorithms that are suggested in the literature.We then discuss current methods to evaluate and compare performance of di?erent crawlers.Finally,we outline the use of Web crawlers in some applications.
2Building a Crawling Infrastructure
Figure1shows the?ow of a basic sequential crawler(in section2.6we con-sider multi-threaded crawlers).The crawler maintains a list of unvisited URLs called the frontier.The list is initialized with seed URLs which may be pro-vided by a user or another program.Each crawling loop involves picking the next URL to crawl from the frontier,fetching the page corresponding to the URL through HTTP,parsing the retrieved page to extract the URLs and ap-plication speci?c information,and?nally adding the unvisited URLs to the frontier.Before the URLs are added to the frontier they may be assigned a score that represents the estimated bene?t of visiting the page corresponding to the URL.The crawling process may be terminated when a certain number of pages have been crawled.If the crawler is ready to crawl another page and the frontier is empty,the situation signals a dead-end for the crawler.The crawler has no new page to fetch and hence it stops.
Crawling can be viewed as a graph search problem.The Web is seen as a large graph with pages at its nodes and hyperlinks as its edges.A crawler starts at a few of the nodes(seeds)and then follows the edges to reach other nodes. The process of fetching a page and extracting the links within it is analogous to expanding a node in graph search.A topical crawler tries to follow edges that are expected to lead to portions of the graph that are relevant to a topic.
4G.Pant,P.Srinivasan,F.Menczer
end C r a w l i n g L o o p Fig.1.Flow of a basic sequential
crawler
2.1Frontier
The frontier is the to-do list of a crawler that contains the URLs of unvisited pages.In graph search terminology the frontier is an open list of unexpanded (unvisited)nodes.Although it may be necessary to store the frontier on disk for large scale crawlers,we will represent the frontier as an in-memory data structure for simplicity.Based on the available memory,one can decide the maximum size of the frontier.Due to the large amount of memory available on PCs today,a frontier size of a 100,000URLs or more is not exceptional.Given a maximum frontier size we need a mechanism to decide which URLs to ignore when this limit is reached.Note that the frontier can ?ll rather quickly as pages are crawled.One can expect around 60,000URLs in the frontier with a crawl of 10,000pages,assuming an average of about 7links per page [30].
The frontier may be implemented as a FIFO queue in which case we have a breadth-?rst crawler that can be used to blindly crawl the Web.The URL to crawl next comes from the head of the queue and the new URLs are added to the tail of the queue.Due to the limited size of the frontier,we need to make sure that we do not add duplicate URLs into the frontier.A linear search to ?nd out if a newly extracted URL is already in the frontier is costly.One solution is to allocate some amount of available memory to maintain a separate hash-table (with URL as key)to store each of the frontier URLs for fast lookup.The hash-table must be kept synchronized with the actual frontier.A more time consuming alternative is to maintain the frontier itself as a hash-table (again with URL as key).This would provide fast lookup for avoiding duplicate URLs.However,each time the crawler needs a URL to crawl,it would need to search and pick the URL with earliest time-stamp (time when a URL was added to the frontier).If memory is less of an issue
Crawling the Web5 than speed,the?rst solution may be preferred.Once the frontier reaches its maximum size,the breadth-?rst crawler can add only one unvisited URL from each new page crawled.
If the frontier is implemented as a priority queue we have a preferential crawler which is also known as a best-?rst crawler.The priority queue may be a dynamic array that is always kept sorted by the estimated score of unvisited URLs.At each step,the best URL is picked from the head of the queue.Once the corresponding page is fetched,the URLs are extracted from it and scored based on some heuristic.They are then added to the frontier in such a manner that the order of the priority queue is maintained.We can avoid duplicate URLs in the frontier by keeping a separate hash-table for lookup.Once the frontier’s maximum size(MAX)is exceeded,only the best MAX URLs are kept in the frontier.
If the crawler?nds the frontier empty when it needs the next URL to crawl,the crawling process comes to a halt.With a large value of MAX and several seed URLs the frontier will rarely reach the empty state.
At times,a crawler may encounter a spider trap that leads it to a large number of di?erent URLs that refer to the same page.One way to alleviate this problem is by limiting the number of pages that the crawler accesses from a given domain.The code associated with the frontier can make sure that every consecutive sequence of k(say100)URLs,picked by the crawler, contains only one URL from a fully quali?ed host name(https://www.doczj.com/doc/585332644.html,). As side-e?ects,the crawler is polite by not accessing the same Web site too often[14],and the crawled pages tend to be more diverse.
2.2History and Page Repository
The crawl history is a time-stamped list of URLs that were fetched by the crawler.In e?ect,it shows the path of the crawler through the Web starting from the seed pages.A URL entry is made into the history only after fetching the corresponding page.This history may be used for post crawl analysis and evaluations.For example,we can associate a value with each page on the crawl path and identify signi?cant events(such as the discovery of an excellent resource).While history may be stored occasionally to the disk,it is also maintained as an in-memory data structure.This provides for a fast lookup to check whether a page has been crawled or not.This check is important to avoid revisiting pages and also to avoid adding the URLs of crawled pages to the limited size frontier.For the same reasons it is important to canonicalize URLs(section2.4)before adding them to the history.
Once a page is fetched,it may be stored/indexed for the master appli-cation(such as a search engine).In its simplest form a page repository may store the crawled pages as separate?les.In that case,each page must map to a unique?le name.One way to do this is to map each page’s URL to a compact string using some form of hashing function with low probabil-ity of collisions(for uniqueness of?le names).The resulting hash value is
6G.Pant,P.Srinivasan,F.Menczer
used as the?le name.We use the MD5one-way hashing function that pro-vides a128bit hash code for each URL.Implementations of MD5and other hashing algorithms are readily available in di?erent programming languages (e.g.,refer to Java2security framework5).The128bit hash value is then converted into a32character hexadecimal equivalent to get the?le name. For example,the content of https://www.doczj.com/doc/585332644.html,/is stored into a?le named160766577426e1d01fcb7735091ec584.This way we have?xed-length ?le names for URLs of arbitrary size.(Of course,if the application needs to cache only a few thousand pages,one may use a simpler hashing mechanism.) The page repository can also be used to check if a URL has been crawled before by converting it to its32character?le name and checking for the exis-tence of that?le in the repository.In some cases this may render unnecessary the use of an in-memory history data structure.
2.3Fetching
In order to fetch a Web page,we need an HTTP client which sends an HTTP request for a page and reads the response.The client needs to have timeouts to make sure that an unnecessary amount of time is not spent on slow servers or in reading large pages.In fact we may typically restrict the client to download only the?rst10-20KB of the page.The client needs to parse the response headers for status codes and redirections.We may also like to parse and store the last-modi?ed header to determine the age of the document.Error-checking and exception handling is important during the page fetching process since we need to deal with millions of remote servers using the same code.In addition it may be bene?cial to collect statistics on timeouts and status codes for identifying problems or automatically changing timeout values.Modern programming languages such as Java and Perl provide very simple and often multiple programmatic interfaces for fetching pages from the Web.However, one must be careful in using high level interfaces where it may be harder to?nd lower level problems.For example,with Java one may want to use the https://www.doczj.com/doc/585332644.html,.Socket class to send HTTP requests instead of using the more ready-made https://www.doczj.com/doc/585332644.html,.HttpURLConnection class.
No discussion about crawling pages from the Web can be complete without talking about the Robot Exclusion Protocol.This protocol provides a mech-anism for Web server administrators to communicate their?le access poli-cies;more speci?cally to identify?les that may not be accessed by a crawler. This is done by keeping a?le named robots.txt under the root directory of the Web server(such as https://www.doczj.com/doc/585332644.html,/robots.txt).This ?le provides access policy for di?erent User-agents(robots or crawlers).A User-agent value of‘*’denotes a default policy for any crawler that does not match other User-agent values in the?le.A number of Disallow entries may be provided for a User-agent.Any URL that starts with the value of a 5https://www.doczj.com/doc/585332644.html,
Crawling the Web7 Disallow?eld must not be retrieved by a crawler matching the User-agent. When a crawler wants to retrieve a page from a Web server,it must?rst fetch the appropriate robots.txt?le and make sure that the URL to be fetched is not disallowed.More details on this exclusion protocol can be found at https://www.doczj.com/doc/585332644.html,/wc/norobots.html.It is e?cient to cache the access policies of a number of servers recently visited by the crawler.This would avoid accessing a robots.txt?le each time you need to fetch a URL. However,one must make sure that cache entries remain su?ciently fresh.
2.4Parsing
Once a page has been fetched,we need to parse its content to extract informa-tion that will feed and possibly guide the future path of the crawler.Parsing may imply simple hyperlink/URL extraction or it may involve the more com-plex process of tidying up the HTML content in order to analyze the HTML tag tree(see section2.5).Parsing might also involve steps to convert the ex-tracted URL to a canonical form,remove stopwords from the page’s content and stem the remaining words.These components of parsing are described next.
URL Extraction and Canonicalization
HTML Parsers are freely available for many di?erent languages.They provide the functionality to easily identify HTML tags and associated attribute-value pairs in a given HTML document.In order to extract hyperlink URLs from a Web page,we can use these parsers to?nd anchor tags and grab the values of associated href attributes.However,we do need to convert any relative URLs to absolute URLs using the base URL of the page from where they were retrieved.
Di?erent URLs that correspond to the same Web page can be mapped onto a single canonical form.This is important in order to avoid fetching the same page many times.Here are some of the steps used in typical URL canonicalization procedures:
?convert the protocol and hostname to lowercase.For example,HTTP:// https://www.doczj.com/doc/585332644.html, is converted to https://www.doczj.com/doc/585332644.html,.
?remove the‘anchor’or‘reference’part of the URL.Hence,http:// https://www.doczj.com/doc/585332644.html,/faq.html#what is reduced to https://www.doczj.com/doc/585332644.html,/faq.html.
?perform URL encoding for some commonly used characters such as‘?’.
This would prevent the crawler from treating https://www.doczj.com/doc/585332644.html,.uiowa.
edu/~pant/as a di?erent URL from https://www.doczj.com/doc/585332644.html,/ %7Epant/.
?for some URLs,add trailing‘/’s.https://www.doczj.com/doc/585332644.html, and https://www.doczj.com/doc/585332644.html,/must map to the same canonical form.
The decision to add a trailing‘/’will require heuristics in many cases.
8G.Pant,P.Srinivasan,F.Menczer
?use heuristics to recognize default Web pages.File names such as index.html or index.htm may be removed from the URL with the as-sumption that they are the default?les.If that is true,they would be retrieved by simply using the base URL.
?remove‘..’and its parent directory from the URL path.Therefore,URL path/%7Epant/BizIntel/Seeds/../ODPSeeds.dat is reduced to /%7Epant/BizIntel/ODPSeeds.dat.
?leave the port numbers in the URL unless it is port80.As an alternative, leave the port numbers in the URL and add port80when no port number is speci?ed.
It is important to be consistent while applying canonicalization rules.It is pos-sible that two seemingly opposite rules work equally well(such as that for port numbers)as long as you apply them consistently across URLs.Other canoni-calization rules may be applied based on the application and prior knowledge about some sites(e.g.,known mirrors).
As noted earlier spider traps pose a serious problem for a crawler.The “dummy”URLs created by spider traps often become increasingly larger in size.A way to tackle such traps is by limiting the URL sizes to,say,128or 256characters.
Stoplisting and Stemming
When parsing a Web page to extract content information or in order to score new URLs suggested by the page,it is often helpful to remove commonly used words or stopwords6such as“it”and“can”.This process of removing stopwords from text is called stoplisting.Note that the Dialog7system recog-nizes no more than nine words(“an”,“and”,“by”,“for”,“from”,“of”,“the”,“to”,and“with”)as the stopwords.In addition to stoplisting,one may also stem the words found in the page.The stemming process normalizes words by con?ating a number of morphologically similar words to a single root form or stem.For example,“connect,”“connected”and“connection”are all reduced to“connect.”Implementations of the commonly used Porter stemming algo-rithm[29]are easily available in many programming languages.One of the authors has experienced cases in bio-medical domain where stemming reduced the precision of the crawling results.
2.5HTML tag tree
Crawlers may assess the value of a URL or a content word by examining the HTML tag context in which it resides.For this,a crawler may need to utilize the tag tree or DOM structure of the HTML page[8,24,27].Figure2shows 6for an example list of stopwords refer to https://www.doczj.com/doc/585332644.html,/idom/ir_ resources/linguistic_utils/stop_words
7https://www.doczj.com/doc/585332644.html,
Crawling the Web9 a tag tree corresponding to an HTML source.Thetag forms the root of the tree and various tags and texts form nodes of the tree.Unfortunately, many Web pages contain badly written HTML.For example,a start tag may not have an end tag(it may not be required by the HTML speci?cation),or the tags may not be properly nested.In many cases,thetag or the
tag is all-together missing from the HTML page.Thus structure-based criteria often require the prior step of converting a“dirty”HTML document into a well-formed one,a process that is called tidying an HTML page.8This includes both insertion of missing tags and the reordering of tags in the page. Tidying an HTML page is necessary for mapping the content of a page onto a tree structure with integrity,where each node has a single parent.Hence,it is an essential precursor to analyzing an HTML page as a tag tree.Note that analyzing the DOM structure is only necessary if the topical crawler intends to use the HTML document structure in a non-trivial manner.For example, if the crawler only needs the links within a page,and the text or portions of the text in the page,one can use simpler HTML parsers.Such parsers are also readily available in many languages.html
body
head
h4 title
text text ul
li
a
text
Fig. 2.An HTML
page and the corre-
sponding tag tree
2.6Multi-threaded Crawlers
A sequential crawling loop spends a large amount of time in which either the CPU is idle(during network/disk access)or the network interface is idle(dur-ing CPU operations).Multi-threading,where each thread follows a crawling 8https://www.doczj.com/doc/585332644.html,/People/Raggett/tidy/
10G.Pant,P.Srinivasan,F.Menczer
loop,can provide reasonable speed-up and e?cient use of available bandwidth. Figure3shows a multi-threaded version of the basic crawler in Figure1.Note that each thread starts by locking the frontier to pick the next URL to crawl. After picking a URL it unlocks the frontier allowing other threads to access it.The frontier is again locked when new URLs are added to it.The locking steps are necessary in order to synchronize the use of the frontier that is now shared among many crawling loops(threads).The model of multi-threaded crawler in Figure3follows a standard parallel computing model[18].Note that a typical crawler would also maintain a shared history data structure for a fast lookup of URLs that have been crawled.Hence,in addition to the frontier it would also need to synchronize access to the history.
The multi-threaded crawler model needs to deal with an empty frontier just like a sequential crawler.However,the issue is less simple now.If a thread ?nds the frontier empty,it does not automatically mean that the crawler as a whole has reached a dead-end.It is possible that other threads are fetching pages and may add new URLs in the near future.One way to deal with the situation is by sending a thread to a sleep state when it sees an empty frontier. When the thread wakes up,it checks again for URLs.A global monitor keeps track of the number of threads currently sleeping.Only when all the threads are in the sleep state does the crawling process stop.More optimizations can be performed on the multi-threaded model described here,as for instance to decrease contentions between the threads and to streamline network access.
This section has described the general components of a crawler.The com-mon infrastructure supports at one extreme a very simple breadth-?rst crawler and at the other end crawler algorithms that may involve very complex URL selection mechanisms.Factors such as frontier size,page parsing strategy, crawler history and page repository have been identi?ed as interesting and important dimensions to crawler de?nitions.
3Crawling Algorithms
We now discuss a number of crawling algorithms that are suggested in the literature.Note that many of these algorithms are variations of the best-?rst scheme.The di?erence is in the heuristics they use to score the unvisited URLs with some algorithms adapting and tuning their parameters before or during the crawl.
3.1Naive Best-First Crawler
A naive best-?rst was one of the crawlers detailed and evaluated by the authors in an extensive study of crawler evaluation[22].This crawler represents a fetched Web page as a vector of words weighted by occurrence frequency. The crawler then computes the cosine similarity of the page to the query or description provided by the user,and scores the unvisited URLs on the
Crawling the Web
11
Frontier Fig.3.A multi-threaded crawler model
page by this similarity value.The URLs are then added to a frontier that is maintained as a priority queue based on these scores.In the next iteration each crawler thread picks the best URL in the frontier to crawl,and returns with new unvisited URLs that are again inserted in the priority queue after being scored based on the cosine similarity of the parent page.The cosine similarity between the page p and a query q is computed by:
sim (q,p )=v q ·v p v q · v p
(1)where v q and v p are term frequency (TF)based vector representations of the query and the page respectively,v q ·v p is the dot (inner)product of the two vectors,and v is the Euclidean norm of the vector v .More sophisticated vector representation of pages,such as the TF-IDF [32]weighting scheme often used in information retrieval,are problematic in crawling applications because there is no a priori knowledge of the distribution of terms across crawled pages.In a multiple thread implementation the crawler acts like a best-N-?rst crawler where N is a function of the number of simultaneously running threads.Thus best-N-?rst is a generalized version of the best-?rst crawler that picks N best URLs to crawl at a time.In our research we have found the best-N-?rst crawler (with N =256)to be a strong competitor [28,23]showing clear superiority on the retrieval of relevant pages.Note that the best-?rst crawler keeps the
12G.Pant,P.Srinivasan,F.Menczer
frontier size within its upper bound by retaining only the best URLs based on the assigned similarity scores.
3.2SharkSearch
SharkSearch[15]is a version of FishSearch[12]with some improvements.It uses a similarity measure like the one used in the naive best-?rst crawler for estimating the relevance of an unvisited URL.However,SharkSearch has a more re?ned notion of potential scores for the links in the crawl frontier.The anchor text,text surrounding the links or link-context,and inherited score from ancestors in?uence the potential scores of links.The ancestors of a URL are the pages that appeared on the crawl path to the URL.SharkSearch,like its predecessor FishSearch,maintains a depth bound.That is,if the crawler ?nds unimportant pages on a crawl path it stops crawling further along that path.To be able to track all the information,each URL in the frontier is associated with a depth and a potential score.The depth bound(d)is provided by the user while the potential score of an unvisited URL is computed as: score(url)=γ·inherited(url)+(1?γ)·neighborhood(url)(2) whereγ<1is a parameter,the neighborhood score signi?es the contex-tual evidence found on the page that contains the hyperlink URL,and the inherited score is obtained from the scores of the ancestors of the URL.More precisely,the inherited score is computed as:
inherited(url)=
δ·sim(q,p)if sim(q,p)>0δ·inherited(p)otherwise
(3)
whereδ<1is again a parameter,q is the query,and p is the page from which the URL was extracted.
The neighborhood score uses the anchor text and the text in the“vicinity”of the anchor in an attempt to re?ne the overall score of the URL by allowing for di?erentiation between links found within the same page.For that purpose, the SharkSearch crawler assigns an anchor score and a context score to each URL.The anchor score is simply the similarity of the anchor text of the hyperlink containing the URL to the query q,i.e.sim(q,anchor text).The context score on the other hand broadens the context of the link to include some nearby words.The resulting augmented context,aug context,is used for computing the context score as follows:
context(url)=
1if anchor(url)>0
sim(q,aug context)otherwise
(4)
Finally we derive the neighborhood score from the anchor score and the context score as:
neighborhood(url)=β·anchor(url)+(1?β)·context(url)(5)
Crawling the Web13 whereβ<1is another parameter.
We note that the implementation of SharkSearch would need to preset four di?erent parameters d,γ,δandβ.Some values for the same are suggested by [15].
3.3Focused Crawler
A focused crawler based on a hypertext classi?er was developed by Chakrabarti et al.[9,6].The basic idea of the crawler was to classify crawled pages with categories in a topic taxonomy.To begin,the crawler requires a topic tax-onomy such as Yahoo or the ODP.9In addition,the user provides example URLs of interest(such as those in a bookmark?le).The example URLs get automatically classi?ed onto various categories of the taxonomy.Through an interactive process,the user can correct the automatic classi?cation,add new categories to the taxonomy and mark some of the categories as“good”(i.e.,of interest to the user).The crawler uses the example URLs to build a Bayesian classi?er that can?nd the probability(Pr(c|p))that a crawled page p belongs to a category c in the taxonomy.Note that by de?nition Pr(r|p)=1where r is the root category of the taxonomy.A relevance score is associated with each crawled page that is computed as:
R(p)=
Pr(c|p)(6)
c∈good
When the crawler is in a“soft”focused mode,it uses the relevance score of the crawled page to score the unvisited URLs extracted from it.The scored URLs are then added to the frontier.Then in a manner similar to the naive best-?rst crawler,it picks the best URL to crawl next.In the“hard”focused mode,for a crawled page p,the classi?er?rst?nds the leaf node c?(in the taxonomy)with maximum probability of including p.If any of the parents(in the taxonomy)of c?are marked as“good”by the user,then the URLs from the crawled page p are extracted and added to the frontier.
Another interesting element of the focused crawler is the use of a distiller. The distiller applies a modi?ed version of Kleinberg’s algorithm[17]to?nd topical hubs.The hubs provide links to many authoritative sources on the topic.The distiller is activated at various times during the crawl and some of the top hubs are added to the frontier.
3.4Context Focused Crawler
Context focused crawlers[13]use Bayesian classi?ers to guide their crawl. However,unlike the focused crawler described above,these classi?ers are trained to estimate the link distance between a crawled page and the rele-vant pages.We can appreciate the value of such an estimation from our own 9https://www.doczj.com/doc/585332644.html,
14G.Pant,P.Srinivasan,F.Menczer
browsing experiences.If we are looking for papers on“numerical analysis,”we may?rst go to the home pages of math or computer science departments and then move to faculty pages which may then lead to the relevant papers.A math department Web site may not have the words“numerical analysis”on its home page.A crawler such as the naive best-?rst crawler would put such a page on low priority and may never visit it.However,if the crawler could estimate that a relevant paper on“numerical analysis”is probably two links away,we would have a way of giving the home page of the math department higher priority than the home page of a law school.
The context focused crawler is trained using a context graph of L layers corresponding to each seed page.The seed page forms the layer0of the graph. The pages corresponding to the in-links to the seed page are in layer1.The in-links to the layer1pages make up the layer2pages and so on.We can obtain the in-links to pages of any layer by using a search engine.Figure4depicts a context graph for https://www.doczj.com/doc/585332644.html,/programs/as seed.Once the context graphs for all of the seeds are obtained,the pages from the same layer (number)from each graph are combined into a single layer.This gives a new set of layers of what is called a merged context graph.This is followed by a feature selection stage where the seed pages(or possibly even layer1pages) are concatenated into a single large https://www.doczj.com/doc/585332644.html,ing the TF-IDF[32]scoring scheme,the top few terms are identi?ed from this document to represent the vocabulary(feature space)that will be used for classi?cation.
A set of naive Bayes classi?ers are built,one for each layer in the merged context graph.All the pages in a layer are used to compute Pr(t|c l),the prob-ability of occurrence of a term t given the class c l corresponding to layer l.
A prior probability,Pr(c l)=1/L,is assigned to each class where L is the number of layers.The probability of a given page p belonging to a class c l can then be computed(Pr(c l|p)).Such probabilities are computed for each class. The class with highest probability is treated as the winning class(layer).How-ever,if the probability for the winning class is still less than a threshold,the crawled page is classi?ed into the“other”class.This“other”class represents pages that do not have a good?t with any of the classes of the context graph. If the probability of the winning class does exceed the threshold,the page is classi?ed into the winning class.
The set of classi?ers corresponding to the context graph provides us with a mechanism to estimate the link distance of a crawled page from relevant pages.If the mechanism works,the math department home page will get classi?ed into layer2while the law school home page will get classi?ed to “others.”The crawler maintains a queue for each class,containing the pages that are crawled and classi?ed into that class.Each queue is sorted by the the probability scores(Pr(c l|p)).When the crawler needs a URL to crawl,it picks the top page in the non-empty queue with smallest l.So it will tend to pick up pages that seem to be closer to the relevant pages?rst.The out-links from such pages will get explored before the out-links of pages that seem to be far away from the relevant portions of the Web.
Crawling the Web15
Fig. 4.A context
graph
3.5InfoSpiders
In InfoSpiders[21,23],an adaptive population of agents search for pages rel-evant to the topic.Each agent is essentially following the crawling loop(sec-tion2)while using an adaptive query list and a neural net to decide which links to follow.The algorithm provides an exclusive frontier for each agent.In a multi-threaded implementation of InfoSpiders(see section5.1)each agent corresponds to a thread of execution.Hence,each thread has a non-contentious access to its own frontier.Note that any of the algorithms described in this chapter may be implemented similarly(one frontier per thread).In the origi-nal algorithm(see,e.g.,[21])each agent kept its frontier limited to the links on the page that was last fetched by the agent.Due to this limited memory approach the crawler was limited to following the links on the current page and it was outperformed by the naive best-?rst crawler on a number of evalu-ation criterion[22].Since then a number of improvements(inspired by naive best-?rst)to the original algorithm have been designed while retaining its capability to learn link estimates via neural nets and focus its search toward more promising areas by selective reproduction.In fact the redesigned version of the algorithm has been found to outperform various versions of naive best-?rst crawlers on speci?c crawling tasks with crawls that are longer than ten thousand pages[23].
The adaptive representation of each agent consists of a list of keywords (initialized with a query or description)and a neural net used to evaluate new links.Each input unit of the neural net receives a count of the frequency with which the keyword occurs in the vicinity of each link to be traversed, weighted to give more importance to keywords occurring near the link(and maximum in the anchor text).There is a single output unit.The output of the neural net is used as a numerical quality estimate for each link considered as input.These estimates are then combined with estimates based on the cosine similarity(Equation1)between the agent’s keyword vector and the page containing the links.A parameterα,0≤α≤1regulates the relative
16G.Pant,P.Srinivasan,F.Menczer
importance given to the estimates based on the neural net versus the parent page.Based on the combined score,the agent uses a stochastic selector to pick one of the links in the frontier with probability
Pr(λ)=
eβσ(λ)
λ ∈φ
e
(7)
whereλis a URL in the local frontier(φ)andσ(λ)is its combined score.The βparameter regulates the greediness of the link selector.
After a new page has been fetched,the agent receives“energy”in pro-portion to the similarity between its keyword vector and the new page.The agent’s neural net can be trained to improve the link estimates by predicting the similarity of the new page,given the inputs from the page that contained the link leading to it.A back-propagation algorithm is used for learning.Such a learning technique provides InfoSpiders with the unique capability to adapt the link-following behavior in the course of a crawl by associating relevance estimates with particular patterns of keyword frequencies around links.
An agent’s energy level is used to determine whether or not an agent should reproduce after visiting a page.An agent reproduces when the energy level passes a constant threshold.The reproduction is meant to bias the search toward areas(agents)that lead to good pages.At reproduction,the o?spring (new agent or thread)receives half of the parent’s link frontier.The o?spring’s keyword vector is also mutated(expanded)by adding the term that is most frequent in the parent’s current document.This term addition strategy in a limited way is comparable to the use of classi?ers in section3.4since both try to identify lexical cues that appear on pages leading up to the relevant pages.
In this section we have presented a variety of crawling algorithms,most of which are variations of the best-?rst scheme.The readers may pursue Menczer et.al.[23]for further details on the algorithmic issues related with some of the crawlers.
4Evaluation of Crawlers
In a general sense,a crawler(especially a topical crawler)may be evaluated on its ability to retrieve“good”pages.However,a major hurdle is the problem of recognizing these good pages.In an operational environment real users may judge the relevance of pages as these are crawled allowing us to determine if the crawl was successful or not.Unfortunately,meaningful experiments involving real users for assessing Web crawls are extremely problematic.For instance the very scale of the Web suggests that in order to obtain a reasonable notion of crawl e?ectiveness one must conduct a large number of crawls,i.e.,involve a large number of users.
Secondly,crawls against the live Web pose serious time constraints.There-fore crawls other than short-lived ones will seem overly burdensome to the
Crawling the Web17 user.We may choose to avoid these time loads by showing the user the results of the full crawl—but this again limits the extent of the crawl.
In the not so distant future,the majority of the direct consumers of in-formation is more likely to be Web agents working on behalf of humans and other Web agents than humans themselves.Thus it is quite reasonable to explore crawlers in a context where the parameters of crawl time and crawl distance may be beyond the limits of human acceptance imposed by user based experimentation.
In general,it is important to compare topical crawlers over a large number of topics and tasks.This will allow us to ascertain the statistical signi?cance of particular bene?ts that we may observe across crawlers.Crawler evaluation research requires an appropriate set of metrics.Recent research reveals several innovative performance measures.But?rst we observe that there are two basic dimensions in the assessment process.We need a measure of the crawled page’s importance and secondly we need a method to summarize performance across
a set of crawled pages.
4.1Page importance
Let us enumerate some of the methods that have been used to measure page importance.
1.Keywords in document:A page is considered relevant if it contains some
or all of the keywords in the query.Also,the frequency with which the keywords appear on the page may be considered[10].
2.Similarity to a query:Often a user speci?es an information need as a short
query.In some cases a longer description of the need may be available.
Similarity between the short or long description and each crawled page may be used to judge the page’s relevance[15,22].
3.Similarity to seed pages:The pages corresponding to the seed URLs,are
used to measure the relevance of each page that is crawled[2].The seed pages are combined together into a single document and the cosine simi-larity of this document and a crawled page is used as the page’s relevance score.
4.Classi?er score:A classi?er may be trained to identify the pages that are
relevant to the information need or task.The training is done using the seed(or pre-speci?ed relevant)pages as positive examples.The trained classi?er will then provide boolean or continuous relevance scores to each of the crawled pages[9,13].
5.Retrieval system rank:N di?erent crawlers are started from the same
seeds and allowed to run till each crawler gathers P pages.All of the N·P pages collected from the crawlers are ranked against the initiating query or description using a retrieval system such as SMART.The rank provided by the retrieval system for a page is used as its relevance score
[22].
18G.Pant,P.Srinivasan,F.Menczer
6.Link-based popularity:One may use algorithms,such as PageRank[5]
or HITS[17],that provide popularity estimates of each of the crawled pages.A simpler method would be to use just the number of in-links to the crawled page to derive similar information[10,2].Many variations of link-based methods using topical weights are choices for measuring topical popularity of pages[4,7].
4.2Summary Analysis
Given a particular measure of page importance we can summarize the per-formance of the crawler with metrics that are analogous to the information retrieval(IR)measures of precision and recall.Precision is the fraction of re-trieved(crawled)pages that are relevant,while recall is the fraction of relevant pages that are retrieved(crawled).In a usual IR task the notion of a relevant set for recall is restricted to a given collection or database.Considering the Web to be one large collection,the relevant set is generally unknown for most Web IR tasks.Hence,explicit recall is hard to measure.Many authors pro-vide precision-like measures that are easier to compute in order to evaluate the crawlers.We will discuss a few such precision-like measures:
1.Acquisition rate:In case we have boolean relevance scores we could mea-
sure the explicit rate at which“good”pages are found.Therefore,if50 relevant pages are found in the?rst500pages crawled,then we have an acquisition rate or harvest rate[1]of10%at500pages.
2.Average relevance:If the relevance scores are continuous they can be av-
eraged over the crawled pages.This is a more general form of harvest rate [9,22,8].The scores may be provided through simple cosine similarity or a trained classi?er.Such averages(see Figure6(a))may be computed over the progress of the crawl(?rst100pages,?rst200pages and so on)
[22].Sometimes running averages are calculated over a window of a few
pages(https://www.doczj.com/doc/585332644.html,st50pages from a current crawl point)[9].
Since measures analogous to recall are hard to compute for the Web,authors resort to indirect indicators for estimating recall.Some such indicators are: 1.Target recall:A set of known relevant URLs are split into two disjoint
sets-targets and seeds.The crawler is started from the seeds pages and the recall of the targets is measured.The target recall is computed as:
target recall=|P t∩P c|
|P t|
where P t is the set of target pages,and P c is the set of crawled pages. The recall of the target set is used as an estimate of the recall of relevant pages.Figure5gives a schematic justi?cation of the measure.Note that the underlying assumption is that the targets are a random subset of the relevant pages.
Crawling the Web
19
P c
Fig.5.The performance metric:|P t ∩P c |/|P t |
as an estimate of |P r ∩P c |/|P r |
2.Robustness :The seed URLs are split into two disjoint sets S a and S b .Each set is used to initialize an instance of the same crawler.The overlap in the pages crawled starting from the two disjoint sets is measured.A large overlap is interpreted as robustness of the crawler in covering relevant portions of the Web [9,6].
There are other metrics that measure the crawler performance in a manner that combines both precision and recall.For example,search length [21]mea-sures the number of pages crawled before a certain percentage of the relevant pages are retrieved.
Figure 6shows an example of performance plots for two di?erent crawlers.The crawler performance is depicted as a trajectory over time (approximated by crawled pages).The naive best-?rst crawler is found to outperform the breadth-?rst crawler based on evaluations over 159topics with 10,000pages crawled by each crawler on each topic (hence the evaluation involves millions of pages).
a
b 0.02 0.025 0.03 0.035 0.04 0 2000 4000 6000
8000 10000a v e r a g e p r e c i s i o n pages crawled Breadth-First Naive Best-First 0 5
10 15 20 25 0 2000 4000 6000 8000 10000
a v e r a g e t a r g e t r e c a l l (%)pages crawled Breadth-First Naive Best-First Fig.6.Performance Plots:(a)average precision (similarity to topic description)(b)average target recall.The averages are calculated over 159topics and the error bars show ±1standard error.One tailed t-test for the alternative hypothesis that the naive best-?rst crawler outperforms the breadth-?rst crawler (at 10,000pages)generates p values that are <0.01for both performance metrics.
20G.Pant,P.Srinivasan,F.Menczer
In this section we have outlined methods for assessing page importance and measures to summarize crawler performance.When conducting a fresh crawl experiment it is important to select an evaluation approach that provides a reasonably complete and su?ciently detailed picture of the crawlers being compared.
5Applications
We now brie?y review a few applications that use crawlers.Our intent is not to be comprehensive but instead to simply highlight their utility.
5.1MySpiders:query-time crawlers
MySpiders[26]is a Java applet that implements the InfoSpiders and the naive best-?rst algorithms.The applet is available online.10Multi-threaded crawlers are started when a user submits a query.Results are displayed dynamically as the crawler?nds“good”pages.The user may browse the results while the crawling continues in the background.The multi-threaded implementation of the applet deviates from the general model speci?ed in Figure3.In line with the autonomous multi-agent nature of the InfoSpiders algorithm(section3.5), each thread has a separate frontier.This applies to the naive best-?rst algo-rithm as well.Hence,each thread is more independent with non-contentious access to its frontier.The applet allows the user to specify the crawling al-gorithm and the maximum number of pages to fetch.In order to initiate the crawl,the system uses the Google Web API11to obtain a few seeds pages. The crawler threads are started from each of the seeds and the crawling con-tinues until the required number of pages are fetched or the frontier is empty. Figure7shows MySpiders working on a user query using the InfoSpiders algorithm.
5.2CORA:building topic-speci?c portals
A topical crawler may be used to build topic-speci?c portals such as sites that index research papers.One such application developed by McCallum et al.[20] collected and maintained research papers in Computer Science(CORA).The crawler used by the application is based on reinforcement learning(RL)that allows for?nding crawling policies that lead to immediate as well as long term bene?ts.The bene?ts are discounted based on how far away they are from the current page.Hence,a hyperlink that is expected to immediately lead to a relevant page is preferred over one that is likely to bear fruit after a few links.The need to consider future bene?t along a crawl path is motivated 10https://www.doczj.com/doc/585332644.html,
11https://www.doczj.com/doc/585332644.html,/apis
什么是淘宝直通车，具体怎么做直通车 淘宝直通车是淘宝上的一种收费推广方式，按点击率来扣费的，这个能把你店铺的宝贝展示到买家搜索的第一页，效果很不错，但是也很烧钱；新店不建议做直通车，因为大多数新店都会亏钱；可以等店铺有一钻信誉后再尝试做淘宝直通车试试。 如有不懂的问题可以来咨询娟娟老师，娟娟老师可随时为你解答各种网店相关的疑问。 想开网店的话可以加娟娟老师微信或QQ，娟娟老师免费教新手开网店 如何找到娟娟老师的联系方式： （在电脑上的话，点击右侧【进入官网】即可看到娟娟老师的QQ和微信） （在手机上的话，点击左下角【访问官网】即可看到娟娟老师的QQ和微信） （“进入官网”旁边的电话是我的手机号，由于打电话的人太多，无法一一接听，所以请大家加我微信交谈， 手机号就是我的微信号） 自我介绍下：我叫黎娟娟，江苏南京人，89年的，大家都叫我娟娟老师。本人到目前为止网店已经开了有八九年了，经验非常丰富，收入也颇丰，每个月都有三万以上收入。现在我主要当网店老师专门教新手开网店。（当初我也是从新手一步步过来的，从最初月收入两千多，到第二个月的五千多，到第三个月的近一万，再到现在每月稳定在三万以上，经历了很多风雨，并积累了丰富经验）所以我很清楚新手如何才能把网店开成功。想开网店的话可以加我哦，免费教新手开网店。 附上一张本人照片，让大家认识下 开网店有两个关键：①找到稳定可靠的货源；②做好店铺的推广营销和活动；打算开淘宝网店的话，要把重点放在找货源和做推广营销上面！关于推广营销这个方面，大家可以加娟娟老师QQ或微信，来我这边学习经验，免费提供教学。 至于货源的话，由于大多数新手自己都没有货源，所以我在这篇文章下面重点跟新手们讲讲如何找货源。其实找货源并不难，但关键是要找到稳定可靠的货源才行！那怎样才能找到稳定可靠的货源呢？为了很好的解决这个问题，娟娟老师推荐新手使用商为开店软件来提供货源，为何要推荐用这个软件提供货源？下面跟大家详细介绍下这个软件作用就知道了【需要软件的话请联系娟娟老师】。
淘宝直通车新手入门教程 l新手入门第一课――广告位与竞价词 一.广告位 让我们来亲身感受一下什么是直通车的广告!按我的步骤来一起操作一下哦,Go 1. 首先打开淘宝首页,在搜索框输入”风衣”这个词,点击搜索按钮,显示搜索页面 2. 往右上角看,有一个掌柜热荐的位置,下面有5个广告位,这是直通车的广告位 3. 把页面拉到最底端,会看到三个大图,这三个也是直通车的广告位 以上的步骤可以演示为下图,红色框的为直通车广告位. 1. 2.
3. 二.竞价词 这些卖家的广告为什么会出现在这里呢? 因为他们都设置了风衣这个竞价词 那竞价词又是什么呢? 就是买家输入这个词搜索，你的广告就能出现。 就像百度的搜索,如果信息符合被搜索的关键词,这条信息就会出现,在直通车,我们把这个关键词称为竞价词. 比如你希望买家输入“风衣”这个词，他就可以在我们的广告位上看见你的宝贝，那么“风衣”就是你要设置的竞价词。
新手入门第二课——收费与排名原则 大家都知道,直通车是一个收费的产品,那到底是怎么收费呢? 多少钱一天还是有包月还是其他收费方式呢? 直通车不是按时间收费的,它的收费方式是:按点击收费 广告展示在广告位上了,我们不收费,只有当买家对您的宝贝感兴趣,点击了您的宝贝,才会有费用产生.所以广告展示跟时间无关，只和余额、日最高限额和定时投放有关（第三课有详细讲解）。 点一次多少钱呢? 每次点击最少1毛钱 那最多呢? 最多多少钱是您自己设置的,您设置的高,扣的钱就多,设置的低就扣的少. 大家都喜欢设置的低,可以少扣点,那设置的高和低有什么区别呢?(排名规则) 比如”风衣”这个词,有20个人买了这个词,但是第一页只有5个人广告位,谁排在前面呢?这时候就需要看谁对”风衣”这个词的出价高,出价越高,排位越前,当然排位越前的每次点击扣的费用也越多. 这个出价就是竞价词的价格 扣钱是从我的支付宝账户扣还是有什么其他方式呢? 是从直通车账户扣款的,首次充值直通车最少500元,按点击扣费,没有任何服务费用,也没有使用期限
在对淘宝直通车的运作模式和基础操作有了一定的了解后，就该进行淘宝直通车实战了。一大把服装圈为网友们带来《淘宝直通车技巧篇》，希望可以让广大网友更好的掌握淘宝直通车的技巧从而更好的进行推广活动。 我们知道，直通车搜索的原则是当卖家设置的词和买家搜索的词完全一样的时候，才会展示宝贝的广告。所以说，给宝贝设置竞价词是至关重要的。直接影响到您的推广效果。有的掌柜会问，那我该怎么设置竞价词？设置竞价词的思路是什么呢？ 淘宝直通车技巧篇：设置竞价词的思路 设置竞价词一定要站在买家的角度去考虑，您要买这件宝贝的适合，会用些什么样的词搜索。要把浏览量大的词和浏览量小的词结合起来推广。浏览量大的词排名不要很前面（除非产品很有优势），浏览量小的词一定要排在前面，否则出现的机会就更少了。 设置竞价词的基本原则是：您要从买家的角度去考虑，如果我是买家，我要搜索这件宝贝要输入哪些关键词呢？
淘宝直通车技巧篇：设置竞价词的思路 首先，第一点，宝贝名称，从您宝贝的名称中提炼出来关键词来作为宝贝的竞价词。 第二点，宝贝详情里的属性词，宝贝详情是我们在编辑宝贝信息的时候抓取出来的关键信息，也是买家十分关注的，所以说用宝贝详情里的属性词作为宝贝的竞价词是十分明智的。 第三点，名称词和属性词里面的组合词。这些词相对比较精确，买家的购买欲望也十分强。 淘宝直通车技巧篇：设置竞价词的思路 总结了设置竞价词的思路，我们再来看一个例子。图中展示的是一件韩版风衣，它的宝贝详情已经给大家列出来了。包括它的价格，颜色，品牌以及风格。各位掌柜，您看到这件宝贝的话你会设置哪些竞价词呢？
淘宝直通车技巧篇：设置竞价词的思路 首先，第一点，宝贝的名称词中我们可以用“风衣”这个竞价词。 第二点，宝贝详情里面的属性词，我们可以用双排扣、韩版、淑女、绿色、长款等等作为竞价词。 第三点，在宝贝名称和宝贝详情的组合词中，我们可以用韩版风衣，双排扣风衣等作为关键词。
开拼多多场景推广的话，首先要有基础销量，还要有个不错的转化率。场景推广是很容易产生爆款的，曝光也高，但前提是你对拼多多各方面有所了解、有一定推广基础才行。 概说： 首先，我们先弄明白拼多多场景推广的展示以及扣费规则： 排名规则： 综合排名=商品质量分广告出价。 商品质量分=点击率转化率销量交易额。 扣费规则： 扣费=（下一位的出价*下一位的商品素材点击率）/自己的商品素材点击率+0.01元。 单次点击扣费，重复点击虚假点击系统会过滤，不计扣费。 定向： 1. 全体人群：所有普通用户 2. 访客重定向：浏览或购买过我的店内商品的用户。 3. 相似商品定向：浏览或购买过相似商品的用户。 4. 叶子类目定向：近期有推广商品所属叶子类目行为的用户。 5. 相似店铺定向：近期对我的店铺的竞品店铺感兴趣的用户。 6. 兴趣点：近期对我的商品相关属性感兴趣的用户。(最多设置5个定向点）。 资源位： 1. 基础流量包：默认包含以下3个展示资源位 2. 类目商品页：推广商品将展示在拼多多商城类目标签页、搜索标签页下方的商品列表中 3. 商品详情页：推广商品将展示在拼多多商城商品详情页为你推荐下方的商品列表中（相似商品） 4. 营销活动页：推广商品将展示在拼多多营销活动页面下方的商品列表中，包括多多果园、边逛边赚、现金签到页、天天领现金、拼多多微信公众号； ---开始正题--- 一. 排名权重与优化： 1. 场景一样有排名权重区分的。如何获得一个号的排名，这个就需要针对商品做出一定的优化。并且要了解场景排名权重的核心环节。 场景排名核心： 场景计划权重--开设每一个计划都有一定的计划权重分，具体是按照改个计划内所有商品的质量分与投入计算所得。 商品质量分--通俗的说法按照以下权重划分：点击率—转化率—产出—订单量—产出比。------这里面对于出价的标准就看你商品的质量分是否够高。 上述两个点是最为基础并且最主要的两个核心，只要懂这些核心内容才能提高场景的排名。 2. 优化推广内容： 计划以5天为一个优化周期，将所有定向与兴趣点5个选择，资源位全选；溢价标准以每个所需推广位置皆有曝光。分别记录每个的曝光量，点击数，点击率，订单量和投产比。5天结束后，记录下每日的点击率、转化率。横向对比你的点击率。根据记录的数据去分析，将曝光量大且点击数点击率高的组合开设一个新的计划。 二．实操： 1. 将优化做好后，直接进入降低出价的步骤。 上述说过排名权重的几个要点，其中我们需要注意的是点击率，这里因为有了上面的数据，
1：定位和利润决定淘宝店的生死！ 这个定位和利润写在所有营销之前，因为这是重中之重！(产品是最基本核心和基础） 基本上可以一句话说：店铺的定位决定你的店铺生死！ 我所说的定位分两种： 1.定位市场细分化 2.定位低端还是中高端客户 何谓细分市场，这是一个比较大的话题，涉及到区域，客户特殊需求，性别，年龄，职业… 这里不说这些，我简单的举两个例子，对大家理解更好更有帮助。 例如做减肥产品—中药减肥产品—针对女性中药减肥产品—针对产后女性中药减肥产品… 为什么要定位细分市场？ 1.细分市场竞争更低更能满足客户需求 2.细分市场跳出了同质化 3.细分市场才是打造网货品牌的地方 这里说的另一个定位直接简单分为定位低端客户还是定位中高端客户利润就不用说了，高利润和低利润，很简单定位中高端客户直接意味着高利润 我要求大家的是尽量定位中高端客户！ 为什么要定位中高端客户？ 1.定位中高端客户的高利润为你以后的推广路子！低端客户的低利润直
接卡死你以后的推广路子，直通车，钻石展位，超级麦霸，聚划算，谷歌竞价广告（和这些基本绝缘）淘宝数据显示2010年电子商务的流量成本是以前的4倍，直接体现在淘宝！25元直通车点击 2.你的低端客户低价策略，在淘宝这个变态的平台总有比你更低的没有最低只有更低，求低价客户需要的是更低价的 3.低端客户要求更多，相反中高端客户反而更好伺候 4.中高端客户的回头购买次数更多 5.中高端客户带给你关键的东西更多利润 6.做中高端客户才有可能打造出网货品牌（七格格麦包包绿盒子…)不可能要求所有店铺都定位中高端，但是有一点是肯定的，你定位低端的话，基本前途很有限了。尤其是淘宝流量不断从C 店剥夺的大势下。淘宝的流量价格在不断的飙升！ 我所希望的是至少看了我这个课程的朋友尽量好好想想你自己店铺的定位，看是否要做出改变！ 2：淘宝营销的本质是什么？！ 淘宝营销的本质到底是什么？ 很多人说流量，有人说营销推广，总感觉不够本质 淘宝购物客户的特点是什么？ 对比购买，从一大堆找到的宝贝中选择一个自己中意的成交,正因为淘宝购物客户的购物对比的天性基因，所以我们卖家的营销也一定要对比的
那些被高手隐瞒的直通车绝密技巧 上官七七现在淘宝直通车开车的技巧，很多的人都讲过，综合所述，内容无非就是设置关键词，以及提高质量得分，而在开车的过程中有很多的观点其实是错误的。而这些错误的观点其实就是开车油费太厉害的原因。 首先我们需要明白的一点是直通车的本质，在淘宝直通车的介绍里面清楚的写着，客户精准营销工具，什么叫精准营销？精准营销就是把我们的信息准确的呈现在需要的顾客面前，这里最重要的一点是需要的顾客。而这个过程中就是通过关键词来实现信息的准确传达。 那么在精准营销的本质上，关键词的数量是一个关键。很多的人都有这么一个观点，那就是关键词越多越好，直通车每一款产品理论上来说可以上800个关键词，这个也是官方教程所宣传的，那么在实际的操作过程中，如果一件产品真的上了800个关键词，那么结果肯定是要亏损的。而且你会发现直通车的转化率非常的低，为什么会出现这种情况呢，因为这种大量的上关键词违背了直通车精准营销的本质。（开车的思路，思路决定出路） 一件产品能够承受多少关键词这个是屈指可数的，在精准营销的基础上我们要考虑的因素非常的多。例如顾客的爱好，年龄，搜索习惯，产品的用料，做工，款式，细节，品牌，季节，气温等等的因素，只有那些符合各种因素的关键词，才是我们需要的关键词，如果我们用各种因素来赛选一下自己的关键词会发现，能够符合的关键词非常的少，通常不会超过100个，只有考虑到各种因素所筛选出来的关键词才是符合直通车精准营销本质的。(直通车词数量) 那么如果我们不考虑这些因素，只是单纯的增加关键词的数量那么结果是显而易见的，那就是大量和宝贝无关的关键词会被点击，造成车费的无端浪费，和转换率的低下，从某一个方面来说，淘宝官方论坛里面
华尔思华为安全HCIE直通车课程 安全HCIE直通车课程涵盖了目前华为安全方向的所有初级高级课程知识，总共分为HCNA（4天）－NCNP（8天）－HCIE（6天）三个阶段，内容安排由浅入深，适合所有零安全基础的学生参加。 一、HCNA课程简介 课程覆盖网络安全基础知识，防火墙基础知识，包过滤技术、NAT技术等防火墙基本原理以及在华为防火墙中的实现，华为防火墙用户管理及认证原理，IPSec、SSL等VPN技术原理以及在华为防火墙中的实现，UTM技术及相关防御策略的部署配置，终端安全技术及基本安全策略配置。 HCNA课程知识大纲：防火墙基础 1.网络安全基本概念 2.防火墙基本概念、防火墙功能特性、防火培设备管理和防火墙基础配置 3.包过滤技术基础、防火墙转发原理、防火墙安全策略及应用 4.网络地址转换技术基础、基于源IP地址NAT技术与配置、基于目的IP地址NAT技术与配置、双向NAT技术与配置、NAT应用场景配置 5.防火墙双机热备技术基础、防火墙双机热备基本组网及配置 6.防火墙用户管理基础、用户认证概念、AAA技术原理、用户认证管理及应用 7.VLAN技术基础、WLAN技术基础、广域网接口特性技术基础VPN基础技术
1.VPN基本概念、VPN分类、加密技术 2.L2TP技术原理、Client-Initialized方式L2TP技术与配置 3.GRE技术原理与配置 4.IPSec基本概念、AH技术原理、ESP技术原理、IKE技术原理、IPSec配置 5.SSL技术原理、虚拟网关概念与配置、Web代理技术与配置、文件共享技术与配置、端口转发技术与配置、网络扩展技术与配置 二、HCNP课程简介 HCNP课程内容覆盖以下四个方面： 1）防火墙通用技术（防火墙基础技术、防火墙安全策略、防火墙用户管理、VPN、安全配置文件、攻击防范、虚拟化和带宽策略） 2）终端安全体系规划、部署、维护与优化 3）安全解决方案和规划设计方案 4）安全体系架构和安全标准的最佳实践。 2. 课程知识大纲 防火墙基础技术 1.防火墙安全策略技术 2.防火墙NAT技术 3.防火墙用户管理技术 4.双机热备典型组网" 5.虚拟防火墙技术 6.防火墙带宽管理技术"
直通车报告 一、关于直通车概念 想要做好直通车，就要先去了解现在直通车的定义，这块很容易被忽视，但若不说，很多“无知”的老板还是在看别人直通车如何如何做成的爆款，如何如何挣钱。停留在过去希望直通车打爆款丶直通车赚钱的年代。并不是说直通车现在打不了爆款赚不了钱，只是有这想法之前，要先明白自己提供什么产品。开车之前要做好哪些相应的准备。 直通车的作用：现在的直通车只是点评运营手段之一，只是一个工具，但是他是使用最多的工具，现在的直通车更偏向与新品引流测款，在新店或者店铺效果不太好的适合，可以快速通过直通车提升店铺浏览。并且提升单品搜索关炎键词的排名。现在的直通车基本上每家淘宝店铺都在做，而且有特色的产品越来越多，所以现在的直通车更偏向于，让原先有销量基础的商品（所谓的爆款）销量更好。让没有销量的产品获得一定的流量。所以，在看待数据的时候需要更加理性客观。 了解直通车在推广操作中所遇到的问题：直通车的问题基本上总结出来：展现量、点击率、点击单价、ROI、转换率以及相对无线端。大家都很清楚，无线的流量基本上能占店铺总流量的70%以上。大家对无线的需求也就越来越多。 二、直通车的推广手段： 很多人了解了直通车的皮毛，就开始希望能做好直通车，做不好了就会开始抱怨直通车太贵，淘宝赚不到钱。这种抱怨除了说服自己放弃直通车以外，解决不了任何问题。 想了解直通车，先看看直通车的推广手段： 关键词推广：流量精准、直击买家 定向推广：流量大，点击单价低，相对来说ROI比较高（不适合新品） 店铺推广：有门槛，费用低，流量精准性差 三、选款 有很多大神都有单独分析选款，之前也对选款做了一些分析。今天不做重点，主要提一些点给大家： 产品自身基础： 1、价格 2、销量和评价（有一些基础评价，毕竟敢于第一个吃螃蟹的人终归是少数） 3、参谋分析：1. 宝贝在店铺中的流量趋势2. 转换率3. 收藏、加购等 市场竞争力，市场趋势 如果说宝贝的质量、价格、卖点上没有一项在同行中有竞争力，那么推广也只是为同行产品做广告而已。首先一定要对自己做的产品类目非常了解，清楚的知道同行产品有什么优势，然后确定自己的产品定位，是否有卖点更突出，或者性价比更高？在同行产品中一定要有优势才值得去开车付费推广。 充足的货源，如果推广起来，之后因为缺货而终止了，那就实在是太可惜了。 产品的利润空间，个人认为起码利润要在30%以上才比较适合开车，利润在30%那么ROI需要做到3.33以上才能保证直通车是盈亏平衡的一个状态。大家可以算一下自己的产品需要什么样的ROI 才能满足最基本的盈亏平衡。 产品的质量，产品的质量要对得起这个价格，不然起步的情况下都是中差评，之后的推广将无法进行，买家不死傻子，不要想以次充好，产品的质量影响我们是否能长久的推广下去！ 四、上车准备 1. 详情页 上车之前宝贝的详情页一定要做好充足的优化，详情页是影响宝贝转换的非常重要的因素，不然开车即使引入很好，很精准的流量不能造成转化，那也是白白浪费钱，时间和精力，尤其像现在无线端，一定要单独的去做一份。图片不需要很多，但是一定要清晰，加载速度够快。产品的卖点突出描写。让客户引起兴趣，抓住需求点，引导下单。 关联营销，买家进店要充分的利用每一个买家，关联营销也很重要。 2. 宝贝优化
目录 项目一：《快乐购物行》 (1) 一、实训目的 (1) 二、实训理论基础 (1) 三、实训内容 (1) 四、实训要求 (1) 五、实训过程或步骤 (1) （一）注册淘宝 (1) （二）注册支付宝 (2) （三）网上购物 (2) （四）收藏宝贝 (3) 六、实训成果或考核办法 (4) 项目二：《淘宝网开店》 (5) 一、实训目的 (5) 二、实训理论基础 (5) 三、实训内容 (5) 四、实训步骤 (7) （一）、开店流程 (7) （二）、网上开店的流程 (9) （三）、淘宝开店建议： (10) （四）、网上开店，魅力何在？ (10)
（五）、网上店铺技巧 (11) 五、实训要求 (14) 六、实训成果与考核 (15) 项目三：《网上商店推广》 (16) 一、实训目的 (16) 二、实训理论基础 (16) 三、实训内容 (16) 四、实训要求 (16) 五、实训过程或步骤 (16) （一）设定好宝贝的关键字 (16) （二）搜索引擎 (17) （三）论坛 (17) （四）促销活动 (18) （五）人脉关系 (18) （六）一元拍 (19) （七）友情链接 (19) （八）论坛广告位 (19) （九）雅虎直通车 (20) 六、实训成果或考核办法 (22) 七、实训总结 (22)
项目一：《快乐购物行》 一、实训目的 要求学生熟练掌握互联网网上购物的专业流程及其应用知识与技能。 二、实训理论基础 《电子商务基础与应用》与《网络技术与应用》两门专业核心课程 中相关理论知识。 三、实训内容 能够熟练运用互联网网上购物的专业流程及其应用知识与技能，以个人为单位完成淘宝网, 百度有啊,拍拍网用户注册、支付账号申请、搜索宝贝、收藏宝贝、联络卖家、出价、评价等专业流程，并完成实训过程记录。 四、实训要求 要求学生借助电子商务创业公司中相关先进设备和自身业余时间熟练运作互联网网上购物的专业流程及其应用知识与技能，做到人人达到上述实训目的。 五、实训过程或步骤 （一）注册淘宝 1．淘宝网的网址为（）。 2．注册淘宝时的步骤为：（填写会员信息）、（填入验证码）、（填入手机号码）。 3．注册完成后淘宝用户名为（），注册时使用的EMAIL为（）。 注意：用户名的字符个数有限制，为（）个字符，密码为（）
这下小类目卖家要笑了，关于直通车定向的福利爆料！ 说到定向啊，相比大家经常使用的直通车关键词投放，直通车定向对大家来说陌生的多。但是陌生不代表不好用，特别是用的人少意味着竞争度低，如果大家玩好了直通车定向，收益那是大大滴有啊！ 这堂课，我就要为大家详细讲讲那个神秘莫测的直通车玩法——直通车定向！ 一、定向的基础 相比关键词投放，定向投放的适用面是比较窄的，所以定向的基础其实是对投放产品的选择，选择对了合适和产品，定向投放起来事半功倍，而如果选错了投放的产品，基本上过就是在浪费钱了。 定向适合投放感性消费的产品，这和客户的购物习惯有关。定向投放不同于关键词投放，投放的人群是比较泛的，是非精准投放，在非精准投放中，低价位的感性消费商品是转化率比较高的。 并且定向不适用商品推广的前期，定向推广的宝贝需要有一定的销量基础，在宝贝还没一定销量的时候，定向进来的流量是很难转化的。 1.如何选择投放的产品 选择投放产品我们看三个维度：消费人群，价格区间，销量。 消费人群 和关键词投放不同，定向投放针对的消费人群是感性消费人群。也就是说，适合投放的商品是感性消费商品，如零食、女装、化妆品、书籍等。而不适合投放理性消费的商品，如电脑、手机、家具、灯具等。这是为什么呢？这里举个栗子说明吧。 美女果果今天打算在淘宝上买点衣服，于是果果搜索了【prada正品新款】这时候跳出来了一堆的衣服的信息，于是她就开始找啊找啊，比啊比啊。突然，果果发现一堆衣服信息里面出现了一个【零食】的主图，是她非常爱吃的夏威夷果，而且价格也挺便宜的，果果就点进
去看了，并且买了两斤。买了夏威夷果之后，果果觉得既然开始买零食了，那就索性多买点吧，于是又买了一堆牛油果啊，奇异果啊等等等。 最后，美女果果没有买到原来打算买的衣服，反而买了一堆零食，并且花了一整个晚上在购买零食上面。 果果美女的消费过程，就是一个非常典型的感性消费者消费过程，她们在购物过程中非常容易受到目标之外的感性消费产品的影响，从而产生购买行为。但是如果在购物过程中看到的是：日常用品、电脑、手机等理性消费商品，她们被影响的概率同样不是很高。 我们再举个例子 这次是帅哥石头，打算在淘宝上为手机买一条长一点的数据线，于是石头就搜索了【iPhone6数据线】，又在搜完出现的信息中按销量做了一下降序排列，快速浏览了一遍，了解了一下行情的价格。在流量过程中，扫到了一个【电脑】的主图，但是由于不是这次的购买目标，被石头直接略过了。在大致看了一下后，石头很快的选定了一家销量和口碑都错的店铺，下单购买了数据线，关闭淘宝退出了，整个购物流程只有5分钟。 石头帅哥的消费过程无疑就是非常理性的，他在购物过程中非常具有目的性，购物时间短暂。而在购物过程中，电脑、手机，日常用品这样的理性消费商品是完全无法干扰到他们。而零食、衣服等感性消费商品有一定几率吸引到他们的注意力，但是效果肯定也没有对感性消费人群的效果那么好。 通过我们上面两个例子，相信大家都能明白，我们在进行定向投放时，需要针对感性消费人群投放感性消费商品的信息，这样才能获得最好的效果。 价格区间 容易让消费者产生感性消费的商品，价格肯定是不会太高的。感性消费者在消费中低价位的商品时，是比较冲动而盲目的，但是如果面对的是高价格的商品，再感性的消费者也会恢复一定的理性，起码需要衡量一下自己钱包里面的票票够不够。 所以，定向投放时适合投放中低价位的商品，这样才能获得比较不错的转化率。 这其实也很好理解，大家都有一时冲动，买了件衣服，买了条裤子的经历，但是谁特么一时冲动买过一套房子？所以当面对高单价的商品时，再感性的人都会回归理性，毕竟没钱是木有感性的资本的。（简单的说，就是没钱没法任性。）
淘宝自带营销工具介绍 1、直通车 淘宝直通车是为淘宝卖家量身定制的，按点击付费的效果营销工具，实现宝贝的精准推广。 功能： 在给宝贝带来曝光量的同时，精准的搜索匹配也给宝贝带来精准的潜在买家。 优点： 1.多：多维度、全方位提供各类报表以及信息咨询，为推广宝贝打下坚实的基础。 2.快：快速、便捷的批量操作工具，让宝贝管理流程更科学、更高效。 3.好：智能化的预测工具，制定宝贝优化方案时更胸有成竹，信心百倍。 4.省：人性化的时间、地域管理方式，有效控制推广费用，省时、省力、更省成本！ 采用直通车该如何做 1．您想推广某一个宝贝，就要为该宝贝设置相应的竞价词及广告标题、简介。
2．当买家来淘宝任何地方搜索了你设置的竞价词，或者点击了你宝贝的类目的时候，您的广告就会出现，展示在搜索结果页最上方的右侧、及最下方。 3．如果买家点了您的直通车广告，系统就会根据您设定竞价词的点击价格来扣费，每次点击最低0.01元。如广告只是展示，没人点击，是不计费的。 2、团购 团购（group purchase）就是团体购物。指认识或不认识的消费者联合起来，加大与商家的谈判能力，以求得最优价格的一种购物方式。 优势： 1．对于没有多少时间的顾客或者一些要求比较高的顾客是有价格上的好处的（这类顾客在论坛上比较少）。 2.对于一天到晚都在网络上学习知识或者有一定时间去了解市场的顾客来说，意义就是你会找建材市场便宜的服务好的商家，不太可能把每个品种找完的，总有你找不到的，找不到的就有团购就可以去参加，虽然不一定便宜，不过也是不贵的，也许这个价格和特价是一样的。 3.对于已经确定了一个固定品牌的产品，而这个品牌基本上没有经销商，只有自营店，那么团购也还是有用的。因为你已经固定了品牌，好不好都是那个了，而只有自营店，又找不到经销商杀价，就算你讲价讲得到团购的价格，也要花很多的精力了。
网站优化的重要性 常常会听到一个声音，“我的策略是薄利多销”。依托供应链的相对优势，主动实施降价促销，通过单品走量。与此同时，也会听到一种抱怨“淘宝定价太低，价格高了卖不出去”，于是一而再，再而三地降低价格。这两个问题可以归结为一个问题：定价。 目前，淘宝卖家主流的新品定价方式分为：经验定价法，根据过去的经验，对宝贝进行定价。依靠店主，或是核心团队开会，对不同产品价格进行判断。好处是省时省力，快速决策，谈笑之间，价格已定。但风险在于，对市场把握一旦出现问题，要么宝贝很难卖出去，要么就是没有赚到钱;成本加成法，新产品进货的价格加上一些，就是宝贝的价格。比如进货价100元的棉衣，加上80%，定价就成了180元;竞争定价法，参考下竞争对手的定价。然后再拍拍脑袋，加一点或减一点，于是价格就出来了。 新品定价的基本前提是要保证盈利。对于任何一个店铺来说：扩大生意规模，需要投入采购资金。随着交易规模的持续扩大，团队人员数量与质量的提升，也意味在薪酬方面更大的投入;再加上直通车、钻石展位等推广投资;还有办公场所、图片拍摄等硬件投入。所以，定价的目的在于保证利润，同时在此前提下可以促进销量的增长。 定价的经济学基础 摸着石头过河，这“石头”就是经济学原理，把几十年积淀下来的原理作为咱们过河的基石，少走弯路。市场的销售商，定价决策依据的是供需关系和如何解决供需平衡的问题。 很多卖家都有同样的经历，在淘宝上，相同的宝贝，越便宜，买的人越多。同理，相同的宝贝利润越高，卖得人也越多。 而如何可以达到一款宝贝，卖的人多，买的人也多，同时卖家有利可图，而买家又可以接受产品的价格?这也就是所说的供需平衡的问题。需要在产品定价之前，就进行核算。 大家都知道，降价是有成本的。举个例子，每件男士衬衫定价是100元的时候，可以卖出1000件，成交是10000元。但降价到80元，原来就能卖出的100件，每件就损失了20元。那么潜在的损失就是(100-80)×100=2000元。如果要达到原来的交易额10000元，就必须卖出1250 件。只有达到1250件以上的时候，总成交才会因降价而上升。反之，就算销量增加，但卖不出125件的时候，成交反而因降价而下降。在竞争中，基于不同商品的不同价格弹性，进行产品组合销售而产生盈利的也有很多成功的案例。 比如，亚马逊花了5.4亿美金收购尿布电子商务的网站https://www.doczj.com/doc/585332644.html,，而这家网站2010年营业额约为3亿美元。它把一类本不在网上售卖的商品推上了网络销售，因为纸尿布不仅需要快速运输，而且标准化程度高，价格透明。价格弹性也比较高，只要有更低的价格，买家会很快转移到其他的购物平台，所以利润被压得很低。但是，他们并非依靠纸尿布来盈利，网上销售纸尿布只是用来吸引一批忠诚的顾客。利润来源于一些网上售卖的高利润率的商品，比如婴儿沐浴液、湿纸巾等。通过销售不同价格弹性商品组合，来实现盈利。 因此一味降价和提价都需要一个合理的考量，也就是找到需求的平衡点。假设价格越低，购买意愿越强，但无法量化购买者会多买多少。于是，你需要制定一个均衡的价格，可以达到利润和销量的同比增长。就是当你提价，销量的下降比例低于提价幅度，那么支付宝成交非但不会下降，反而会上升，而且这个时候，利润会更加好看。 如何让你的宝贝卖高价? 信息不对称是指“一些成员拥有其他成员无法拥有的信息”。具体点来说，就是淘宝买家对产品的了解没有卖家多，面对海量的信息，他们只能根据掌握的有限信息对宝贝做出购买决策。在此前提下，就容易出现因为交易双方信息不对称和市场价格下降产生的劣质品驱逐优质品，形成了盲目的价格战，而使得消费者开始以价格作为唯一判断标准，相同的宝贝，哪个便宜就买哪个，使得真正有品质差别的宝贝销售不出去。
淘宝直通车 淘宝直通车是由阿里巴巴集团下的雅虎中国和淘宝网进行资源整合，推出的一种全新的搜索竞价模式。他的竞价结果不只可以在雅虎搜索引擎上显示，还可以在淘宝网（以全新的图片+文字的形式显示）上充分展示。每件商品可以设置200个关键字，卖家可以针对每个竞价词自由定价，并且可以看到在雅虎和淘宝网上的排名位置，并按实际被点击次数付费淘宝(每个关键词最低出价0.05元最高出价是100元，每次加价最低为0.01元)。 一、淘宝直通车的准入规则介绍 B店(商城店)：商城卖家可不受级别限制即可加入。 C店(卖家普通店)： 1、集市店铺只有卖家级别达到两颗心，11个好评。 2、店铺非虚拟交易近半年的DSR评分三项指标不得低于4.4（开店不足半年的从开店之日起算）。 3、店铺好评率不得低于97%。 4、以下几个主营类目的卖家需要先加入消保才能开通直通车： (1)保健品/滋补品 (2)古董/邮币/字画/收藏 (3)母婴用品/奶粉/孕妇装 (4)品牌手表/流行手表 (5)食品/茶叶/ 零食/特产 (6)腾讯QQ专区 二、开通淘宝直通车的7个步骤： 第1步：首先是进入淘宝创想开通直通车业务，进入网页之后点击页面上的加入直通车。 第2步：然后会弹出淘宝直通车服务协议，点击确定完成后续操作之后就可以开通直通车服务了。
第3步：如果上面的条件你符合且接受了，然后你可以登陆淘宝直通车后台管理(主页面上有一个直接登陆后台）点击推广宝贝。 第4步：选择你需要推广的宝贝并且添加必要的描述，以便能吸引到顾客。
第5步：最重要的则是关键词设置，关键词决定了你在搜索中被别人搜到的可能性大小。关键词设置的原则是：热门，贴切，日常常使用。具体关键词设置可以参考系统推荐或者网上一些关键词设置教程。这一步是比较重要的一步。 第6步：然后则是设置推广价格：直通车推广是根据点击量计算花费，每产生一次点击累积一次竞价价格。当然你要明白两点：有点击不一定有购买，购买远少于点击量。所以要保证你的商品够吸引力。第二点：有竞价不一定有展示，既然是竞价模式，则是按照使用者竞价高低给予展示机会的，所以选择合适的竞价也很重要。
现在淘宝直通车开车的技巧，很多的人都讲过，综合所述，内容无非就是设置关键词，以及提高质量得分，而在开车的过程中有很多的观点其实是错误的。而这些错误的观点其实就是开车油费太厉害的原因。 首先我们需要明白的一点是直通车的本质，在淘宝直通车的介绍里面清楚的写着，客户精准营销工具，什么叫精准营销？精准营销就是把我们的信息准确的呈现在需要的顾客面前，这里最重要的一点是需要的顾客。而这个过程中就是通过关键词来实现信息的准确传达。 那么在精准营销的本质上，关键词的数量是一个关键。很多的人都有这么一个观点，那就是关键词越多越好，直通车每一款产品理论上来说可以上800个关键词，这个也是官方教程所宣传的，那么在实际的操作过程中，如果一件产品真的上了800个关键词，那么结果肯定是要亏损的。而且你会发现直通车的转化率非常的低，为什么会出现这种情况呢，因为这种大量的上关键词违背了直通车精准营销的本质。 一件产品能够承受多少关键词这个是屈指可数的，在精准营销的基础上我们要考虑的因素非常的多。例如顾客的爱好，年龄，搜索习惯，产品的用料，做工，款式，细节，品牌，季节，气温等等的因素，只有那些符合各种因素的关键词，才是我们需要的关键词，如果我们用各种因素来赛选一下自己的关键词会发现，能够符合的关键词非常的少，通常不会超过100个，只有考虑到各种因素所筛选出来的关键词才是符合直通车精准营销本质的。 那么如果我们不考虑这些因素，只是单纯的增加关键词的数量那么结果是显而易见的，那就是大量和宝贝无关的关键词会被点击，造成车费的无端浪费，和转换率的低下，从某一个方面来说，淘宝官方论坛里面大量宣传多关键词的作用其实就是让大家多花点钱，而淘宝就能够多赚一点。
直通车推广优化技巧，全程实操干货教程 今天聊聊直通车质量分怎么优化，提升质量分。 我开直通车这么久了，用过三个方法是比较有效的 第一种 1.选词的时候就选择10-20个以创意标题的且属于宝贝属性相符的分词作为关键词。然后把展现量低的，点击率低的，转化率还差的删掉，还有展现指数和转化率还在下滑的也删掉 这里说一下，我们也可以在直通车后台竞争分析，就可以知道我们的数据和和用户数据的差距的，有对比才知道我们应该着重去优化哪个数据 2.移动端的关键词排名保持在10-20名，靠前的也肯定也是可以的，看我们的预算来决定，太后了效果太差了，不能快速的提升质量分（权重），电脑端的排名，我现在都不怎么关注 3.投放平台只投放站内无线端，投放地域的话，只有是看点击率和转化率，比如我们投前1 5的地区 在点击率和转化率综合考虑选出15个或者是10个地区来投放就可以
投放时间，主要是看我们的后天数据表现，如果没有数据的话，就投白天就可以。后面，数据多了，再去优化。原则是那个时间段访客多就多投，转化率高的就多投，少的就相反。 投放人群的话，有数据就按照原来的数据投，进行溢价。如果是没有的话，还是按照我们之前的20%进行溢价，后面再优化，一会下面再展开。 4.投放后就可以进行优化，当关键词展现量有100以上的时候就要去看点击率，看跟同行的点击率相差多少，是高于还是低了，低了就删掉，不要，否则肯定会拉低权重。如果某个关键词表现比较优秀，就进行溢价，人群也是，表现的比同行好很多也要进行溢价。 比如我发现“手提包女气质女神”这个关键词表现很好，那我就有可能多加几个与“气质女神”相关的词，如果这个词表现很差，那我就把相关的都删了。 5.如果我在一个不熟悉的类目，就会先海选，把选50个关键词左右，人群溢价5%，然后根据前面的方法进行优化。测试好关键词、人群、主图、出价之后，重新创建一个计划，重新按照测试好的进行投放，把原来测试的计划删除。因为测试肯定是改来改去的，这样数据就可能比较乱，不好分析，而且权重不好做不起 第二种方法： 找大量精准的长尾词，三四级词的长尾词，特别是近期都在上升的。只要确保这个长尾词可以找到我们宝贝的就可以用，而且竞争指数低的、市场均价也低的词 这些词的展现指数一般在1000-2000左右的，具体看类目，只要这些词够多，也会得到很大的展现
淘宝直通车运营操作技巧 现在淘宝直通车开车的技巧，很多的人都讲过，综合所述，内容无非就是设置关键词，以及提高质量得分，而在开车的过程中有很多的观点其实是错误的。 而这些错误的观点其实就是开车油费太厉害的原因。首先我们需要明白的一点是直通车的本质，在淘宝直通车的介绍里面清楚的写着，客户精准营销工具，什么叫精准营销,精准营销就是把我们的信息准确的呈现在需要的顾客面前，这里最重要的一点是需要的顾客。 而这个过程中就是通过关键词来实现信息的准确传达。那么在精准营销的本质上，关键词的数量是一个关键。很多的人都有这么一个观点，那就是关键词越多越好，直通车每一款产品理论上来说可以上800个关键词，这个也是官方教程所宣传的，那么在实际的操作过程中，如果一件产品真的上了800个关键词，那么结果肯定是要亏损的。而且你会发现直通车的转化率非常的低，为什么会出现这种情况呢，因为这种大量的上关键词违背了直通车精准营销的本质。 一件产品能够承受多少关键词这个是屈指可数的，在精准营销的基础上我们要考虑的因素非常的多。例如顾客的爱好，年龄，搜索习惯，产品的用料，做工，款式，细节，品牌，季节，气温等等的因素，只有那些符合各种因素的关键词，才是我们需要的关键词，如果我们用各种因素来赛选一下自己的关键词会发现，能够符合的关键词非常的少，通常不会超过100个，只有考虑到各种因素所筛选出来的关键词才是符合直通车精准营销本质的。那么如果我们不考虑这些因素，只是单纯的增加关键词的数量那么结果是显而易见的，那就是大量和宝贝无关的关键词会被点击，造成车费的无端浪费，和转换率的低下，从某一个方面来说，淘宝官方论坛里面大量宣传多关键词的作用其实就是让大家多花点钱，而淘宝就能够多赚一点。
直通车推广技巧 直通车操作技巧现在淘宝直通车开车的技巧，很多的人都讲过，综合所述，内容无非就是设置关键词，以及提高质量得分，而在开车的过程中有很多的观点其实是错误的。而这些错误的观点其实就是开车油费太厉害的原因。 首先我们需要明白的一点是直通车的本质，在淘宝直通车的介绍里面清楚的写着，客户精准营销工具，什么叫精准营销,精准营销就是把我们的信息准确的呈现在需要的顾客面前，这里最重要的一点是需要的顾客。而这个过程中就是通过关键词来实现信息的准确传达。那么在精准营销的本质上，关键词的数量是一个关键。很多的人都有这么一个观点，那就是关键词越多越好，直通车每一款产品理论上来说可以上800个关键词，这个也是官方教程所宣传的，那么在实际的操作过程中，如果一件产品真的上了800个关键词，那么结果肯定是要亏损的。而且你会发现直通车的转化率非常的低，为什么会出现这种情况呢，因为这种大量的上关键词违背了直通车精准营销的本质。 一件产品能够承受多少关键词这个是屈指可数的，在精准营销的基础上我们要考虑的因素非常的多。例如顾客的爱好，年龄，搜索习惯，产品的用料，做工，款式，细节，品牌，季节，气温等等的因素，只有那些符合各种因素的关键词，才是我们需要的关键词，如果我们用各种因素来赛选一下自己的关键词会发现，能够符合的关键词非常的少，通常不会超过100个，只有考虑到各种因素所筛选出来的关键词才是符合直通车精准营销本质的。那么如果我们不考虑这些因素，只是单纯的增加关键词的数量那么结果是显而易见的，那就是大量和宝贝无关的关键词会被点击，造成车费的无端浪费，和转换率的低下，从某一个方面来说，淘宝官方论坛里面大量宣传多关键词的作用其实就是让大家多花点钱，而淘宝就能够多赚一点。
此文章出处富网店https://www.doczj.com/doc/585332644.html, 淘宝直通车基础设置篇 以前和大家一样都是从苦逼的淘宝民工开始，09年到现在一直操作直通车，很多人把直通车神化了（对新手来说）在网络上很多新手咨询了我很多关于直通车的问题，今天有幸写一篇淘宝直通车基础设置给新手看。 理清思路，不要看太多被神化的直通车文章，直通车只是一个工具。 关于一开始直通车的设置，我们可以看到一个计划里面有设置日限额设置投放平台设置投放时间设置投放地域 一开始，当你选好款后投放在一个计划里面，很多新手不知道怎么设置，也是让很多新手头疼的问题，其实很简单，一开始选款后，里面的设置，例如： 设置日限额，你可以按照自己的情况进行设置
设置投放平台：在设置里面可以看到两大分类：淘宝站内，淘宝站外 很多买家一开始都是直接全部都开通，其实不然，淘宝会给出这样的一个设置一定有他的道理，还有关于阶段性的设置，宝贝测试情况，一般淘兵开车都是站外直接关闭，只保留淘宝站内搜索推广，Why？很简单，当一款宝贝前期没做过任何测试的前提下，你不知道这款宝贝的大概趋势，流量，PPC，点击率，转化率
刚刚说到阶段性设置，于是乎，会有人问我什么是阶段性设置，其实很容易理解，就是直通车前期测试图片点击转化还可以，当销量达到一定程度，比如小爆的情况下，转化率一定时，加大流量，提升销量。 设置投放时间： 还是那句话，当宝贝没做过任何测试的话，你很难知道你自己宝贝的表现情况怎么样，所以全部设置全部时间段，100%投放，只是前期的设置，后期根据这款宝贝的表现再做调整， 比如你这款宝贝在某个时间段，流量一般，转化一般，可以设置折扣低点
淘宝直通车推广秘籍 直通车作为为淘宝卖家量身定制的营销工具，为卖家实现宝贝的精准推广。简单来说，直通车在给宝贝带来曝光量的同时，精准的搜索匹配也给宝贝带来了精准的潜在买家。对于淘宝卖家而言，淘宝直通车推广是一个多维度，全方位为商品进行推广的工具，并且拥有快速、便捷、智能、个性化等功能。其实，对于许多卖家而言，虽然知道直通车的存在和优势，但是却不知道该如何利用推广。那么，今天JA就和大家分享一些关于直通车推广的技巧吧。 直通车的推广手段： 很多人了解了直通车的皮毛，就开始希望能做好直通车，做不好了就会开始抱怨直通车太贵，淘宝赚不到钱。这种抱怨除了说服自己放弃直通车以外，解决不了任何问题。 想了解直通车，先看看直通车的推广手段： 关键词推广：流量精准、直击买家 定向推广：流量大，点击单价低，相对来说ROI比较高（不适合新品） 店铺推广：有门槛，费用低，流量精准性差 选款 有很多大神都有单独分析选款，之前也对选款做了一些分析。今天不做重点，主要提一些点给大家： 产品自身基础： 1、价格 2、销量和评价（有一些基础评价，毕竟敢于第一个吃螃蟹的人终归是少数） 3、参谋分析：1. 宝贝在店铺中的流量趋势2. 转换率3. 收藏、加购等 市场竞争力，市场趋势 如果说宝贝的质量、价格、卖点上没有一项在同行中有竞争力，那么推广也只是为同行产品做广告而已。首先一定要对自己做的产品类目非常了解，清楚的知道同行产品有什么优势，然后确定自己的产品定位，是否有卖点更突出，或者性价比更高？在同行产品中一定要有优势才值得去开车付费推广。
充足的货源，如果推广起来，之后因为缺货而终止了，那就实在是太可惜了。 产品的利润空间，个人认为起码利润要在30%以上才比较适合开车，利润在30%那么ROI需要做到3.33以上才能保证直通车是盈亏平衡的一个状态。大家可以算一下自己的产品需要什么样的ROI才能满足最基本的盈亏平衡。 产品的质量，产品的质量要对得起这个价格，不然起步的情况下都是中差评，之后的推广将无法进行，买家不死傻子，不要想以次充好，产品的质量影响我们是否能长久的推广下去！ 上车准备： 1. 详情页 上车之前宝贝的详情页一定要做好充足的优化，详情页是影响宝贝转换的非常重要的因素，不然开车即使引入很好，很精准的流量不能造成转化，那也是白白浪费钱，时间和精力，尤其像现在无线端，一定要单独的去做一份。图片不需要很多，但是一定要清晰，加载速度够快。产品的卖点突出描写。让客户引起兴趣，抓住需求点，引导下单。 关联营销，买家进店要充分的利用每一个买家，关联营销也很重要。 2. 宝贝优化 开车推广最终目的也是为了提升自然流量，让宝贝有稳定流量来源，宝贝自身的优化也非常重要。包括宝贝自然标题、主图、属性、卖点等方面的优化，有利于直通车推广带动自然流量的提升。 推广： 直通车推广有几个数据是我们要重点观察测试的，根据数据的反馈才能合理有效的对直通车进行优化调整。 1. 展现量 大家应该都知道，直通车培养关键词的重点是点击率，但是点击率的前提要有足够的展现，有些店家开车的时候，不敢出价畏手畏脚的，导致没有展现没有点击，反而应该学整体计划的权重，导致越来越难做，其实在开始出价的的时候还是大方一点，出价比市场高出20%以上，有足够的展现，然后再去根据关键词的表现进行优化调整。还有一个重点就是选词，选什么样的关键词测试，开始大家都知道要选精准词，选几个精准长尾词对前期测试来说是最好的选择，之后开始慢慢加入二级热门词，推广到中后期需要大流量的时候开始加入热词。 2.点击率