Thanks for the response, Vijay. I didn't mean to insinuate that I thought the team was holding back tooling for the public (rather that such tooling wouldn't make sense to release), but it's reassuring to hear that the total TensorFlow experience is pretty much the same internally and externally.
I think part of the conspiracy theorizing is due to the misconceived notion that Google has some "secret sauce" that allows it to do what it does, as opposed to many talented engineers spending a lot of man-hours on a problem. There has also been a fair amount of negative Google sentiment in the community recently, and the story that Google is holding out on developers feeds into this narrative.
Benchmarking has always been low-hanging fruit for community members to latch onto for the sake of attacking/defending a particular framework. However, the practical difference between these frameworks (assuming each is configured properly) seem to be within a margin of error and are constantly changing (not to mention the inconsistencies you mentioned), so choosing a framework solely on its benchmarking scores is narrow-minded.
From what I've seen, benchmarking has been more useful as a discovery mechanism for areas in a codebase that can be improved. The TensorFlow team has done an excellent job of using various benchmarks to guide development, and I imagine other frameworks are doing the same.
I think part of the conspiracy theorizing is due to the misconceived notion that Google has some "secret sauce" that allows it to do what it does, as opposed to many talented engineers spending a lot of man-hours on a problem. There has also been a fair amount of negative Google sentiment in the community recently, and the story that Google is holding out on developers feeds into this narrative.
Benchmarking has always been low-hanging fruit for community members to latch onto for the sake of attacking/defending a particular framework. However, the practical difference between these frameworks (assuming each is configured properly) seem to be within a margin of error and are constantly changing (not to mention the inconsistencies you mentioned), so choosing a framework solely on its benchmarking scores is narrow-minded.
From what I've seen, benchmarking has been more useful as a discovery mechanism for areas in a codebase that can be improved. The TensorFlow team has done an excellent job of using various benchmarks to guide development, and I imagine other frameworks are doing the same.