Security in Agent Systems

1.5 Information from Strangers

It's not called the Net of a Million Lies for nothing. So, you're totally wired. Your company has a T1 line to the Net, you have accounts with three different ISP's, and you have five Web-search engines in your hotlist. And the more information that you can get to, the more uncertain you are which of it to trust. Now consider the plight of an "intelligent" agent, sent out into this morass to find out something for you. Not only is it deluged and overloaded with information, but it's also stupid. Even the most "intelligent" programs today are orders of magnitude less capable than humans, especially in their ability to apply broad world knowledge to particular problems, to grasp wholes, to appreciate subtleties, to evaluate plausibility. If we place too much trust in the intelligence of "intelligent agent" systems, we find ourselves indirectly subject to various new kinds of fraud. If you search on "sex" or its obvious variants on one of the public-access web-search pages, you will find, among other things, pages with titles of the form:

John(Sex) Smith's(Sex) Home(Nude) Page(GIF)

On the page itself, John will explain that there's really nothing here about sex, he just wanted to get lots of visitors. If you search on the names of any of the candidates in the 1996 U.S. Presidential elections, you are likely to find a number of pages. Some are provided by the candidate's organization, others by independent and unofficial supporters, and others by opponents. Especially in the case of "parody" pages, it can be difficult to tell which pages are which. A human, let alone an automatic web-searching agent, might have a hard time getting a reliable and consistent impression of the candidate's stands from this set of information. At present, this sort of thing is not a serious problem. People use web-searching agents only to retrieve a list of pointers to possibly-interesting sites; the human user then (often slowly and painfully) culls through the pointers, ignoring some, following others, adding the best hits to a hotlist, or just recording the data found there. A human will not try to use John (Sex) Smith's home page as a source for naughty pictures, nor form an impression of Robert Dole's opinions based on information from a page whose background is a lattice of Dole brand pineapple cans. But a program might not be so acute. I cannot count on an agent program to venture off into the network to make purchases on my behalf unless it can reliably determine just what is for sale, and how likely it is to satisfy my needs. Especially in the case of information goods, whose content may be complex, subjective, and hard to summarize, agent programs will be (even) easier to mislead than humans. This is a security problem itself, if not one that computer security professionals are used to considering. It also leads indirectly to more familiar concerns: if agents are used to negotiate deals for security-significant resources (which includes everything from time and directory services to routing information), they will need to have some assurance that they are not getting an especially good deal because the seller intends to attack the system by selling it corrupted information. There are a number of plausible approaches to solving this problem:

Agents could be used to gather data only from trusted and controlled sources. But this deprives us of one of the expected benefits of agents: the ability to understand and deal with uncontrolled and non-standardized information.
We could rely on the legal system to punish those who commit fraud by intentionally publishing misleading information to agents. This is cold comfort alone, though; we want the average fraud attempt to fail, not simply to be possibly punishable.
Trusted reputation servers could help agents to determine how much faith to put in information from various sources. There are various technical and social problems to be solved in providing a reputation infrastructure, but it is likely that a mature electronic economy will require it [9].
Trusted evaluation or review services could allow skilled humans to provide a large number of agents with reliable descriptions of goods, in a standard format.

Whatever approaches turn out to be most effective, the danger of agent-targeted fraud will have to be addressed in any mature agent-based information system.

[ Top of Page | Previous Page | Next Page | Table of Contents ]

back to index