Is it allowed to take things from a public place, for eg taking some flowers or leaves or soil from a public park?

If it is not against the law, then no problem.