Start port listeners once plugins have been loaded

Port listeners are currently started even before plugins have been loaded. That means that users may momentarily find the server in a not fully initialized state since some plugins may change some functionalities or add new ones. That is why we want to make sure to start listeners once we tried to load plugins at least once (and only plugins found during the first iteration in PluginManager).