Will the missing internal classes in System.Data.Linq be OSS as well?


#1

This morning I looked at the referencesource repo at github and was delighted that e.g. Linq to Sql (in System.Data.Linq) was present. I cloned the repo locally and tried to compile the assembly but it was not doable as various classes were missing. Looking with reflector at System.Data.Linq it looks like all internal classes in System.Data.Linq are not currently present in the referencesource repo at github.

Are there any plans to add these later or will these stay hidden? It would be great to have those as a full fork of Linq to Sql is now impossible.

For the curious, yes I’m planning to fork Linq to Sql if it’s available in full, to see if I can leverage its excellent Linq provider to replace parts of our own or extend it to other DBs as I have already query engines available for a lot of other DBs in our own ORM (LLBLGen Pro Runtime Framework). Would be a nice ‘let’s see how far we can get’ project, but I can understand MS might not want to open all code up.

Thanks in advance.


#2

Hello Frans,

Can you add some color as to which classes were missing? I was curious to look into this, but did not know where to start looking.


#3

(partly rephrased :slight_smile: )

In Reflector, please open System.Data.Linq.dll from the ReferenceAssemblies\net45 folder. Comparing these classes to the source in the referencesource repo, you’ll see there are many classes missing. E.g. ‘Error’, ‘SR’, ‘DataQuery< T>’, lots of classes in the mapping folder for model data definitions, however some internal classes are there like SortableBindingList< T>.

Because of this it’s not possible to compile the code. It’s a bit unclear why some stuff is missing while a lot of other stuff is there (also internal stuff): is this intentional (as they can’t be opened yet, or aren’t scheduled to be opened) or they missed a couple of files, or the files are elsewhere (as the repo doesn’t have csproj files it might be the file was linked from somewhere else)


#4

To answer my own question, I’ll quote reply tweets from Kean: As the forum system tells me as a new user I can only link to 2 links in a post but refuses to accept a post with 2 links (after I removed 1, had first 3) I’ll quote the tweets below, not link to them :wink:)

@davkean:
@FransBouma Our aim currently is to get what was available for reference source, available on Github.

@davkean
@FransBouma Then we’ll look at individual cases. None of reference source is in a buildable state.

@davkean
@FransBouma Just to set expectations, our biggest focus is getting .NET Core up and running. LinqToSQL has not yet been ported to that.

So it’s currently not possible, perhaps in the (near) future, who knows :slight_smile:


#5

Nice update!

Once the stuff from referencesource is available on github, we would be more than happy to assist on the process of bringing any code that can be incorporated from it to .NET Core.


#6

@FransBouma:

Hmm. Actually all of the internal classes needed for System.Data.Linq are present with the following exceptions:

  • global::ThisAssembly
  • global::FXAssembly
  • global::AssemblyRef
  • System.Global.Linq.SR
  • System.Global.Linq.Error
  • System.Global.Linq.Strings
  • System.Global.Linq.Mapping.SR
  • System.Global.Linq.Mapping.Error
  • System.Global.Linq.Mapping.Strings
  • System.Global.Linq.SqlClient.SR
  • System.Global.Linq.SqlClient.Strings

Also missing is the majority of the partial class:

  • System.Global.Linq.SqlClient.Error

That is literally all that is missing. Now, as for why those are missing? Well every single one of those classes indicated are generated code. The SR and Strings classes are presumable derived from the missing .resx files. I have no idea how they go about generating the Error classes, but they definitely do. The global classes are generated by filling in metadata about the release, and other DLLs compiled as part of the framework. (Although only global::ThisAssembly appears to be referenced. The others might be used by unreleased unit tests, or might just be standardized classes availble in every framework dll, but only used by some.

I have successfully built System.Data.Linq.dll (referencing the official 4.0 framework reference assemblies) by pasting reflector de-compiled versions of those classes in the project, and setting the correct set of Define Directives. (Both ILGEN and SYSTEM_DATA_LINQ must be defined).

I did not actually test that it works, and without the resource files it would probably fail badly if an error occurs and it tries to throw an exception, bu the successful compilation proves that all the other internal classes are there.

Of course, for Mono this might be only of limited use, since it relies heavily on System.Data, and especially System.Data.SqlClient. I am not sure if Mono’s implementation is a close enough match that it could work.

As for using Microsoft’s implementation for Mono, while it appears that all of the C# code for System.Data was released (minus a very similar set of generated files), portions of the code known as SNI were written in C++/CLI and were not released as open source. Interestingly, but not usefully, the C++/CLI source in was published on http://referencesource.microsoft.com/ (although are only visible if you use the download link, since the Rosyln generated site does not understand C++). Obviously they compile the C# code to a netmodule, and then link it in with the unmanaged code using link.exe. The last I knew Mono lacked a C++/CLI compiler.

Hope the above is helpful, or at least interesting.


#7

From the looks of it, many more classes are missing, e.g. all abstract classes in Mapping and LinqtosqlShared.Mapping are missing as well, at least on github there are much less classes, but I’ve to check with a type list whether all types are actually there. That the strings/SR classes are generated is a good suggestion, didn’t think of that!

To my knowledge, Linq to Sql has over 10 million unit tests (that’s a number I heard once, might be wrong!), let’s hope these are opened as well later on.


#8

I also recommend taking a look at EF7 over at (github)[https://github.com/aspnet/EntityFramework). It already has multiple database support, and a SQLite provider to boot, to show how to add support for other databases. Currently, it doesn’t work on anything but linux (simple reason being incompatible libsqlite I’ve been told) but it should at least show you the code required to do query generating etc. for a new DB provider for EF7.


#9

I am aware of the Entity Framework and what they’re doing in EF7, but I’m not interested in their code, for one because some of their architectural choices (e.g. off-loading SQL generation to the ADO.NET providers) are IMHO not what should be done. Linq to Sql otoh, has a simpler provider model internally and should be easier to extend and mend to my needs.

As I said above, I’m looking into forking Linq to Sql to add it to our set of ORM frameworks we ship ourselves as a poco alternative to our own non-poco ORM framework.


#10

The “SR” classes are just partial classes with strings. They seem to be the generated resource names for messages.

With our internal research build, we auto-generated teh SR contents as needed.


#11

OK! I thought: “is it possible that I miss the obvious?” and of course that was the case (don’t laugh ;)). I went back to the code and looked at it in class view and indeed all types are there just as @KevinCathcart said, except for the string / string resource classes. There’s a lot of discrepancy between class filenames and class names, but that’s OK.

So what’s left are indeed the resource files to re-gen the classes and the massive amount of unit tests they have and everything’s set!

So consider this thread ‘solved’. Thanks all for your time and info :smile:


.NET Foundation Website | Blog | Projects | Code of Conduct