Try the below option:
detach("package:vegan", unload=TRUE)
NOTE: You can try ?detach for details on detach
Multiple versions of a package can be loaded once.
To guarantee that all copies are detached, use the below function:
detach_package <- function(abcpkg, character.only = FALSE)
{
  if(!character.only)
  {
    abcpkg <- deparse(substitute(abcpkg))
  }
  search_item <- paste("package", abcpkg, sep = ":")
  while(search_item %in% search())
  {
    detach(search_item, unload = TRUE, character.only = TRUE)
  }
}
If you want to use detach for a single package, then, use the below options:
detach_package(vegan)
OR
detach_package("vegan", TRUE)