This sci-fi thriller follows the adventures of the Men in Black (M.I.B.), the brave operatives of the top-secret non-government agency that polices extraterrestrial alien refugees. The agency operates from an underground base at a Triborough Bridge and Tunnel Authority ventilation station in Battery Park, New York City. MIB members use neuralyzers to erase witnesses' memories of alien sightings. MIB is monitoring about 1500 aliens around the world, most of them in the vicinity of New York City.
One night, Agent K (Tommy Lee Jones) and his partner, D (Richard Hamilton) intercept a truck containing illegal immigrants and Mikey, an extraterrestrial disguised as a human. Mikey starts to run and K is forced to destroy it when it tries to attack a border patrol officer. D, feeling too old, asks K to neuralize him into "retirement". Later, the MIB leader, Agent Zed (Rip Torn), suggests that K should search for a new partner.
Meanwhile, James Darrell Edwards III (Will Smith) is a New York police officer pursuing a man on foot. While chasing the incredibly fast and agile fugitive over the rooftops, Edwards sees the man's irises blink vertically, and realizes he is not human. K arrives at the precinct station, questions Edwards before neuralyzing him, and leaves the cop an MIB business card.
Later, Edwards goes to the secret headquarters and competes with several others to qualify to join the MIB. After the tests, K takes him aside (while the others are neuralyzed) and offers him the position. Edwards accepts and his identity is erased, becoming Agent J.
Suspicious of why extraterrestrials are suddenly leaving the planet, the M.I.B. investigate a farmer named Edgar (Vincent D'Onofrio), who has been acting strangely after an alien craft crashed on his farm. Edgar has been killed and his skin used as a disguise by a "Bug", a member of a giant cockroach-like species that are at war with several other alien races, including the Arquillians.
An Arquillian prince hiding in Brooklyn who is disguised as a human jewelry store owner named Rosenberg is attacked, and tells J as he dies that "the galaxy is on Orion's belt".
M.I.B. informant Frank the Pug (Tim Blaney), a Remoolian disguised as a small lapdog, explains that the missing galaxy is a massive source of energy housed in a small jewel. Edgar the Bug figures out the galaxy is hanging on the collar of Rosenberg's cat Orion, which refuses to leave the prince's body at the morgue. Orion has been taken care of by Dr. Laurel Weaver (Linda Fiorentino). J arrives at the morgue just as Edgar kidnaps Weaver and grabs the galaxy.
The Arquillians deliver an ultimatum to M.I.B. to secure the galaxy within an hour, or they will destroy Earth. Edgar the Bug arrives at the site of two disguised flying saucers, the observation towers of the New York State Pavilion at Flushing Meadows. K and J are close behind and destroy one saucer. The Bug sheds Edgar's skin and swallows J's and K's guns. K tells J to stop the Bug from getting onto the other ship, then taunts the Bug until K too is swallowed.
J infuriates the Bug by crushing cockroaches and saying they are his relatives.
The Bug is blown open from the inside by K, who located his gun in the Bug's stomach. As J and K sit on the ground covered in slime, the insect tries to attack again, but is destroyed by Weaver, using J's gun.
The three return to M.I.B. headquarters and K tells J that he has not been training him as a partner, but as a replacement. J neuralyzes K, using a coma cover story to allow him to return to his civilian life and the young woman he left behind. A few days later, it is revealed that Weaver also joined M.I.B. and is now J's new partner, Agent L.
The camera rapidly pulls back, showing that Earth and the Milky Way galaxy are also inside an alien marble being used in a cosmic game.