Many players looking for this solution. Currently the only way to delete items is to throw them into water. In that way they will be destroyed instantly or after a short while. 

I have build a dumping ground in my main base. It is a deep hole in the ground. Every items which is not needed goes in. Thats how I build a wastebin in Valheim. Once its full I will build some ceilings above the hole and I'm fine.