|
On 4/2/13 1:24 PM, Kingsley Idehen
wrote:
On
4/2/13 9:13 AM, Rodriguez-Muro Mariano (A) wrote:
Dear Kingsley,
I am the main developer of ontop and quest, we met during ISWC
in Boston last
year. Regarding your comments about our benchmarks of ontop and
virtuoso. During
ISWC you made similar remarks regarding our benchmarks and
offered direct
assistance through Virtuoso's support team. We did take your
offer and started
working with Virtuoso's support the team. The support case is
#20228, Garry is our
contact.
During the last 4 months he helped us setup our Virtuoso
instances and verify
our results. We did get
better performance from Virutoso compared to the one you saw in
Boston,
however, the SQL queries produced by Quest still came up ontop
of Virtuoso by a very
big margin in the
tests that we did using BSBM and MySQL 5.5 as a backend. If the
rewriting process
is also taken into account, the difference is smaller, but still
strongly
in favour of ontop and Quest plus MySQL.
I can assure you with a 100% certainty that your benchmark numbers
for Virtuoso are inaccurate simply based on your configuration.
The results where also confirmed by Garry using the machines at
open link.
In the process we also found and reported several bugs that
produced wrong
answers or crashes in Virtuoso RDF Views. Garry contacted the
development
team of virtuoso and said they were working to fix the problem.
Not all the new results are online, however, some of them and
the changes we did to the INI file as recommended by Garry are
located at
http://ontop.inf.unibz.it/?page_id=74
and in the wiki page that is linked there.
In sum, we are surprised with your comment about our benchmarks
and 'the way
we advertise' our work.
Because you shouldn't publish benchmarks without configuration
information. All you needed to do was include the Virtuoso INI
settings (as you did the MySQL settings).
Our best numbers for BSBM performance and scalability come from
the use of RDF Views over SQL. Typically, BSBM benchmarks have
only published our RDF store numbers [1][2], and I don't see your
claims exceeding those. Thus, you are basically claiming that your
product makes a mockery of Virtuoso's best performance
configuration for BSBM.
Also note, there are some new auditable BSBM benchmark results
coming out from the Linked Data Benchmarking Council [1], and I
encourage you to digest the numbers and showcase to everyone how
your product can exceed the numbers that will be unveiled. Again,
the numbers we produce will still be a factor of 5 less than what
we can currently achieve via RDF Views or SQL Data.
Links:
1.
http://wifo5-03.informatik.uni-mannheim.de/bizer/berlinsparqlbenchmark/results/V6/#exploreVirtuoso
-- BSBM results (note: Virtuoso INI is part of the publication)
2.
http://wifo5-03.informatik.uni-mannheim.de/bizer/berlinsparqlbenchmark/results/V6/index.html#comparison
3. http://ldbc.eu/ -- EU funded benchmark project for RDF and
Graph databases .
Kingsley
And here is the independent benchmark report [1] that I referred to
above. Basically, this is the BSBM benchmark scaled between 50 and
150 Billion triples. Note, the rule doesn't change, using RDF Views
atop SQL, the performance will be a factor of 5 times faster since
this particular benchmark is SQL relational DBMS oriented.
Links:
1. http://bit.ly/14ULX2F -- Independent BSBM benchmark report scaled
between 50 and 150 Billion triples .
Kingsley
Greetings,
Mariano
On 4/1/13 12:12 PM, Michel Dumontier
wrote:
ok, how about ontop developed by
Mariano Rodriguez and colleagues:
http://ontop.inf.unibz.it/
it enables one to map OWL-QL ontologies to SQL database
and answer
queries. no conversion required.
If you know these folks well enough, please have them publish
their
Virtuoso INI settings. Publishing benchmarks against a poorly
configured
Virtuoso instance e.g., using its default settings (which
assume limited
memory in the < 2GB range) isn't the way to advertize their
work. The
claimed disparity speaks volumes :-)
(02)
_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
--
Regards,
Kingsley Idehen
Founder & CEO
OpenLink Software
Company Web: http://www.openlinksw.com
Personal Weblog: http://www.openlinksw.com/blog/~kidehen
Twitter/Identi.ca handle: @kidehen
Google+ Profile: https://plus.google.com/112399767740508618350/about
LinkedIn Profile: http://www.linkedin.com/in/kidehen
|
smime.p7s
Description: S/MIME Cryptographic Signature
_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J (01)
|