Compatibility Testing Framework


#1

With .Net becoming open source, community contribution may sometimes cause compatibility breaks, regressions, performance deterioration.

So a framework to test out impact off each PR on the other community projects would be beneficial. Also may be pin point what change might cause regression and on what projects. Also perhaps notify the project on potential regressions and issues on and opt in basis.

Setting ups such a project early would be beneficial for the evolution off .net rather than when the pain points surface.


#2

Wouldn’t this be the unit tests in each individual project? I mean, totally agree that a strong set of tests is crucial … but thankfully it seems these are already in place :slight_smile:

edit, or are you referring to a CI server that automatically runs all the tests when a PR is made?


#3

I think there is another dimension here.

On the surface it would seem as long as the .Net Core is ported to a platform, then an app written to use it would be portable to all ported platform. Although a nice idea, currently that isn’t total true in that there are corner edges of the library that aren’t completely implemented on all platforms. For example, I’m working on the System.IO.Pipes name space and for example, pipes on the *NIX platforms don’t support message mode and if you try to use that mode, you will get exceptions thrown.

This brings up the general issue of the philosophy of the .Net Core library, and that is is it a portability platform, ie does an app running on one platform work on all platforms? Are unsupported platform capabilities supported by simulation in the library (e.g. *NIX pipes don’t support message boundaries but could be simulated) Is there a subset of the library that can make the guarantee?

Or is the library a special binding to the underlying ported platform? Only supports what the underlying platforms supports without doing any simulations of unsupported capabilities.

I personally suspect that most people would vote for the former that is the library supports portability across ported platforms.

Bernie Schoch


#4

I am not confident about “most people” although I suspect there would be a solid desire for a highly-portable case. I think the desire to craft features may trump portability even if intended for specialized, platform-specific use.

The great test may be how the Universal Windows Platform rolls out and matures, https://msdn.microsoft.com/en-us/library/windows/apps/dn958439.aspx. Although UWP is specific for adapting across the Windows ecosystem, there will need to be some arrangement to target other platforms via Mono (with the cooperation of the organization) at the least. The mechanisms might also be suitable for cases such as the *NIX pipes case, etc.

Assuming the UWP mechanisms work well, and further abstractions can be bolted in, there remains the social aspect of designing for portability. I think what that comes down to is willingness to demonstrate the achievement of portability as a measurable quality, and then seeing where the effort to accomplish that can actually take root.


.NET Foundation Website | Blog | Projects | Code of Conduct