Hello!
The intended design is that you should not have to do much, if anything, to manage your assets unless you want to. If you have not encountered a need to do so, it is ok to just not worry about it. Files are only added here if they are intentionally added by a user, or if said file is needed by a plan or layout. We also try to cleanup assets that are no longer needed. Also note that a large chunk of assets are what we consider communal, or in other words they are owned by Chief Architect Software and are automatically distributed to all users via the web. These tend to be assets needed by our default templates.
We are also currently looking at tools to improve the management experience here for those who do want to manage them.