How about returning
cosnt std::set<std::unique_ptr<B>> &? This isn't an ownership transfer.