Saturday, 16 August 2008

Search Engines

Having spend sometime on internet searches over the last few months I'd just like to highlight some areas where common mistakes are made.

the first and by far seems to be the most common is cashing, you can speed your search engine by almost 50% by making the cashing larger, this has both a plus and negative to it.

the plus is the speed of the searches will be much faster for common searches however there will be a trade off as your results might not reflect information indexed only minutes before so you'll have a slight delay in new entry's appearing.

another common mistake is missing stop words, now for those of you that don't know what a stop word it here is a quick explanation.

a stop word is a work that you do not index as a search able word in your search engine this word might appear in your description or name of the site but you do not use it as a searchable work such as a word like "the" can you imagine how many sites that would bring back on the results... nearly if not all sites and as a result would slow your search engine down allot, here are some example search words.

a, about, above, across, after, afterwards, again, against, all, almost, alone, along, already, also, although, always, am, among, amongst, amoungst, amount, an, and, another, any, anyhow, anyone, anything, anyway, anywhere, are, around, as, at, back, be, became, because, become, becomes, becoming, been, before, beforehand, behind, being, below, beside, besides, between, beyond, bill, both, bottom, but, by, call, can, cannot, cant, co, computer, con, could, couldnt, cry, de, describe, detail, do, done, down, due, during, each, eg, eight, either, eleven, else, elsewhere, empty, enough, etc, even, ever, every, everyone, everything, everywhere, except, few, fifteen, fify, fill, find, fire, first, five, for, former, formerly, forty, found, four, from, front, full, further, get, give, go, had, has, hasnt, have, he, hence, her, here, hereafter, hereby, herein, hereupon, hers, herself, him, himself, his, how, however, hundred, i, ie, if, in, inc, indeed, interest, into, is, it, its, itself, keep, last, latter, latterly, least, less, ltd, made, many, may, me, meanwhile, might, mill, mine, more, moreover, most, mostly, move, much, must, my, myself, name, namely, neither, never, nevertheless, next, nine, no, nobody, none, noone, nor, not, nothing, now, nowhere, of, off, often, on, once, one, only, onto, or, other, others, otherwise, our, ours, ourselves, out, over, own, part, per, perhaps, please, put, rather, re, same, see, seem, seemed, seeming, seems, serious, several, she, should, show, side, since, sincere, six, sixty, so, some, somehow, someone, something, sometime, sometimes, somewhere, still, such, system, take, ten, than, that, the, their, them, themselves, then, thence, there, thereafter, thereby, therefore, therein, thereupon, these, they, thick, thin, third, this, those, though, three, through, throughout, thru, thus, to, together, too, top, toward, towards, twelve, twenty, two, un, under, until, up, upon, us, very, via, was, we, well, were, what, whatever, when, whence, whenever, where, whereafter, whereas, whereby, wherein, whereupon, wherever, whether, which, while, whither, who, whoever, whole, whom, whose, why, will, with, within, without, would, yet, you, your, yours, yourself, yourselves.

Now as you can see it list is quite long, now one option is to stop a search for less than three letters this removes the problem with over half the stop words being needed.

Also the design of your search engine can help allot as well, having a master database doing the indexing and many slave databases feed the front end site will greatly better balance the load and keep the response times nice a low.

No comments: