The last time I used SWIG, it generate the most unreadable code I could ever imagine so beware of that. Here is the bottom line -> Binding to C++ from C# is possible, but is a bad idea for a simple reason: name mangling.
Name mangling is something C++ compilers do to make sure that compiled symbols have unique names even if there are overloaded methods (same name), etc. The problem is there is no standard for this so each compiler does it as they see fit. So the thing you need to bind to in C# is different between Windows, Mac, Linux, etc. Furthermore, P/Invoke only allows you to bind in a C style way so it will be very unnatural, even if you can overcome this massive problem.
Bonus: These mangled names even change sometimes between compiler versions without warning.
If you are just wrapping C calls with C++ anyway, it seems like the best thing to do would be to make the same C calls via P/Invoke in C#. If that is not possible then you will need to make a C interface and use that (I’m doing the same thing right now so I wrote a script to generate the C# bindings from C headers to speed things up for me)