OntologySummit2013: Panel Session-05 - Thu 2013-02-14    (3LGZ)

Summit Theme: "Ontology Evaluation Across the Ontology Lifecycle"    (3LH0)

Summit Track Title: Track-D: Software Environments for Evaluating Ontologies    (3LH1)

Session Topic: Software Environments for Evaluating Ontologies - I    (3MF7)

Session Co-chairs: Mr. PeterYim (Ontolog; CIM3) and Dr. MichaelDenny (MITRE) - intro slides    (3MF8)

Panelists / Briefings:    (3MF9)

Archives:    (3MFE)

Abstract:    (3MH6)

OntologySummit2013 Session-05: "Software Environments for Evaluating Ontologies-I" - intro slides    (3MH7)

This is our 8th Ontology Summit, a joint initiative by NIST, Ontolog, NCOR, NCBO, IAOA & NCO_NITRD with the support of our co-sponsors. The theme adopted for this Ontology Summit is: "Ontology Evaluation Across the Ontology Lifecycle."    (3MIH)

Currently, there is no agreed methodology for development of ontologies, and there are no universally agreed metrics for ontology evaluation. At the same time, everybody agrees that there are a lot of badly engineered ontologies out there, thus people use -- at least implicitly -- some criteria for the evaluation of ontologies.    (3MII)

During this OntologySummit, we seek to identify best practices for ontology development and evaluation. We will consider the entire lifecycle of an ontology -- from requirements gathering and analysis, through to design and implementation. In this endeavor, the Summit will seek collaboration with the software engineering and knowledge acquisition communities. Research in these fields has led to several mature models for the software lifecycle and the design of knowledge-based systems, and we expect that fruitful interaction among all participants will lead to a consensus for a methodology within ontological engineering. Following earlier Ontology Summit practice, the synthesized results of this season's discourse will be published as a Communiqué.    (3MIJ)

At the Launch Event on 17 Jan 2013, the organizing team provided an overview of the program, and how we will be framing the discourse around the theme of of this OntologySummit. Today's session is one of the events planned.    (3MH8)

Further to our Track-D Mission, we will, in this 5th virtual panel session of the Summit, start by laying out a plan to survey the landscape of software environments, systems and tools that feature "ontology evaluation." This will be followed by briefings from our panelists who will share with us their experience and insights on how a few (exemplary) software environments/systems/tools support ontology evaluation, and help assure quality in the development of ontology or ontology-driven application. Among them, we will include software environments that have "real needs" to evaluate whether and which ontologies would best meet their needs, and how practitioners are assuring the quality of those ontologies in "real applications'. A Q&A and open discussion among all panelists and participants will follow the briefings.    (3MH9)

More details about this OntologySummit is available at: OntologySummit2013 (homepage for this summit)    (3MHA)

Briefings:    (3MHB)

COLORE is an open repository of first-order ontologies that serves as a testbed for ontology evaluation and integration techniques, and which supports the design, evaluation, and application of ontologies in first-order logic. This talk will explore the different techniques that are used when an ontology is uploaded into COLORE.    (3MNP)

In particular, we consider different workflows (sequences of evaluation steps) that are used to verify and modularize an ontology. Although not all of these ontology evaluation workflows are completely automated, we will discuss current directions for implementation.    (3MHF)

Agenda:    (3MHK)

OntologySummit2013 - Panel Session-05    (3MHL)

Proceedings:    (3MHR)

Please refer to the above    (3MHS)

IM Chat Transcript captured during the session:    (3MHT)

 see raw transcript here.    (3MHU)
 (for better clarity, the version below is a re-organized and lightly edited chat-transcript.)
 Participants are welcome to make light edits to their own contributions as they see fit.    (3MHV)
 -- begin in-session chat-transcript --    (3MHW)
	[08:59] PeterYim: Welcome to the    (3N06)
	 = OntologySummit2013: Virtual Panel Session-05 - Thu 2013-02-14 =    (3N07)
	Summit Theme: Ontology Evaluation Across the Ontology Lifecycle    (3N08)
	* Summit Track Title: Track-D: Software Environments for Evaluating Ontologies    (3N09)
	Session Topic: Software Environments for Evaluating Ontologies - I    (3N0A)
	* Session Co-chairs: Mr. PeterYim (Ontolog; CIM3) and Dr. MichaelDenny (MITRE)    (3N0B)
	Panelists / Briefings:    (3N0C)
	* Dr. MichaelDenny (MITRE) - "Ontology Quality and Fitness: A Survey of Software Support"    (3N0D)
	* Professor MichaelGruninger (U of Toronto) - "Ontology Evaluation Workflow in COLORE"    (3N0E)
	* Ms. JeanneHolm (Data.gov; NASA/JPL) - "Evaluating US Open Data for Discovery, Interoperability, 
	and Innovation"    (3N0F)
	* Mr. GavinMatthews (Vertical Search Works) - "Assuring broad quality in large-scale ontologies"    (3N0G)
	Logistics:    (3N0H)
	* Refer to details on session page at: 
	http://ontolog.cim3.net/cgi-bin/wiki.pl?ConferenceCall_2013_02_14    (3N0I)
	* (if you haven't already done so) please click on "settings" (top center) and morph from "anonymous" to your RealName (in WikiWord format)    (3N0J)
	* Mute control: *7 to un-mute ... *6 to mute    (3N0K)
	* Can't find Skype Dial pad? 
	** for Windows Skype users: it's under the "Call" dropdown menu as "Show Dial pad" 
	** for Linux Skype users: please note that the dial-pad is only available on v4.1 (or later or the earlier Skype versions 2.x,)
	   if the dialpad button is not shown in the call window you need to press the "d" hotkey to enable it.    (3N0L)
	Attendees: Akram, AlanRector, AmandaVizedom, AnatolyLevenchuk, BobSchloss, BobbinTeegarden, 
	CarmenChui, ClarePaul, DaliaVaranka, DanCerys, DmitryBorisoglebsky, DougFoxvog, ElieAbiLahoud, 
	FabianNeuhaus, GavinMatthews, JeanneHolm, JimDisbrow, JoanneLuciano, JoaoPauloAlmeida, JohnBilmanis, 
	KenBaclawski, KevinSimkins, LamarHenderson, LeoObrst, MarcelaVegetti, MarcosMartinezRomero, 
	MatthewWest, MaxPetrenko, MichaelDenny, MichaelGruninger, MikeBennett, MikeDean, MikeRiben, 
	OliverKutz, PaulPope, PeterYim, RamSriram, RichardMartin, ScottHills, SimonSpero, SteveRay, 
	TerryLongstreth, TillMossakowski, ToddSchneider, VictorAgroskin, WillBurns,    (3N0M)
	 == Proceedings: ==    (3N0N)
	[08:51] marcelaVegetti morphed into MarcelaVegetti    (3N0O)
	[09:19] MatthewWest1 morphed into MatthewWest    (3N0P)
	[09:19] anonymous1 morphed into MaxPetrenko    (3N0Q)
	[09:21] anonymous morphed into KevinSimkins    (3N0R)
	[09:23] anonymous morphed into GavinMatthews    (3N0S)
	[09:24] anonymous morphed into MichaelDenny    (3N0T)
	[09:27] anonymous morphed into CarmenChui    (3N0U)
	[09:27] anonymous morphed into MarcosMartinezRomero    (3N0V)
	[09:29] anonymous morphed into JeanneHolm    (3N0W)
	[09:32] anonymous morphed into LamarHenderson    (3N0X)
	[09:33] anonymous morphed into DougFoxvog    (3N0Y)
	[09:39] anonymous morphed into PaulPope    (3N0Z)
	[09:40] SimonSpero1 morphed into SimonSpero    (3N10)
	[09:42] anonymous morphed into LamarHenderson    (3N11)
	[09:42] Clare Paul morphed into ClarePaul    (3N12)
	[09:43] JoanneLuciano: Call for help with software development- is there anyone in the community who 
	knows SADI services? (if not, ok check out http://sadiframework.org) I have a generalized framework 
	approach that can be used for development of ontologies and evaluation, providing context for 
	intrinsic and extrinsic evaluation. If you can help, or know anyone who can, please get in touch 
	with me.    (3N13)
	[09:36] PeterYim: == PeterYim opens the session on behalf of the co-chairs ... see: the [0-Chair] slides    (3N14)
	[09:44] List of members: AlanRector, AmandaVizedom, AnatolyLevenchuk, BobbinTeegarden, CarmenChui, 
	ClarePaul, DaliaVaranka, DmitryBorisoglebsky, DougFoxvog, ElieAbiLahoud, FabianNeuhaus, 
	GavinMatthews, JeanneHolm, JimDisbrow, JoanneLuciano, JoaoPauloAlmeida, JohnBilmanis, KenBaclawski, 
	KevinSimkins, LamarHenderson, LeoObrst, MarcelaVegetti, MarcosMartinezRomero, MatthewWest, 
	MaxPetrenko, MichaelDenny, MichaelGruninger, MikeDean, OliverKutz, PaulPope, PeterYim, RamSriram, 
	RichardMartin, SimonSpero, SteveRay, TerryLongstreth, TillMossakowski, ToddSchneider, 
	VictorAgroskin, anonymous, vnc2    (3N15)
	[09:43] PeterYim: == MichaelDenny presenting ... see: the [1-Denny] slides    (3N16)
	[09:44] anonymous morphed into DanCerys    (3N17)
	[09:44] anonymous morphed into Akram    (3N18)
	[09:50] AmandaVizedom: @MichaelDenny - cutting out all human-required evaluation factors might miss 
	much opportunity. Why not include all factors, noting that there may be much that the evaluation 
	environment can do to *assist* with non-automated evaluation and with the tracking and management of 
	evaluation of all factors?    (3N19)
	[09:54] JoanneLuciano: +1 to Amanda's comment -- use computers to assist humans    (3N1A)
	[09:51] anonymous morphed into WillBurns    (3N1B)
	[09:57] SimonSpero: Not perfect, but not bad: 
	http://www.howto.gov/customer-service/collecting-feedback/basics-of-survey-and-question-design    (3N1C)
	[10:01] SimonSpero: More academic: Tourangeau, R., Rips, L.J., and Rasinski, K. (2000), 
	The Psychology of Survey Response, Cambridge: Cambridge University Press. 
	http://www.amstat.org/sections/srms/ThePsychologyofSurveyResponse.pdf    (3N1D)
	[10:02] SimonSpero: http://www.amazon.com/Psychology-Survey-Response-Roger-Tourangeau/dp/0521576296    (3N1E)
	[11:25] MichaelDenny: @SimonSpero: Thanks for the pointers to survey methodology. I am hoping you 
	will be able to help field this survey. I would like to come up with the simplest of implementations 
	that will allow continuing contributions and visibility of results.    (3N1F)
	[09:57] DougFoxvog: @MichaelDenny: you suggest characterizing breadth, depth, use considerations, 
	etc. of ontologies. Do you have an ontology which can be used to express these features of ontologies?    (3N1G)
	[09:58] TerryLongstreth: Perhaps an element of the Maintenance phase, but I prefer adding 
	Retirement' to include archiving, sequestering, or erasing any component    (3N1H)
	[09:58] BobSchloss: [Apologies - last minute unavoidable conflict came up. I will look over the 3 
	sets of slides from the rest of today tomorrow or next week]    (3N1I)
	[09:58] PeterYim: @BobSchloss: thanks for coming (albeit briefly)    (3N1J)
	[10:01] AmandaVizedom: Suggestion for additional evaluation factors (very important for some common 
	applications): Interpreting slide 8 #10 to mean core logical inference: inference type, along with 
	power (e.g., integrated support for probabilistic reasoning/ uncertainty); integrated lexical 
	coverage (languages, support for NLP); integrated topic-relation coverage (facets, subtopic, support 
	for document classification).    (3N1K)
	[10:01] JoanneLuciano: What are the research questions that the survey seeks to obtain (I didn't see 
	that -maybe I missed it).    (3N1L)
	[10:02] DougFoxvog: MichaelDenny brings up "assess[ing] query performance" for an ontology. Isn't 
	this to a large extent a property of the inference engine used, not necessarily of the ontology?    (3N1M)
	[10:05] TerryLongstreth: @doug: For the extrinsics - Track B - Todd and I have discussed (argued) 
	this. He won. Basically, if we can't see the ontology except as it presents behaviors, the 
	interpreting mechanism (whether a person, inference engine or generated software) is part of the 
	ontology.    (3N1N)
	[10:04] AmandaVizedom: @doug I'd say both (that is, it is a characteristic of a specific ontology in 
	a specific reasoning environment).    (3N1O)
	[10:09] DougFoxvog: re: Amanda's comment regarding bullet point "Assess the inferencing power of an 
	ontology". The inferencing power of an *ontology* would be a feature of the kinds of expressions 
	used: inheritance, argument constraints, properties of relations, existence and types of rules, 
	expression of uncertainty, etc. Support for actually inferencing using such statements would depend 
	upon the inference engine.    (3N1P)
	[10:03] JoanneLuciano: suggest adding an "open question" -- to ask what survey respondents think is 
	important but was not asked on survey.    (3N1Q)
	[10:04] MarcelaVegetti: +1 to Joanne's comment    (3N1R)
	[10:03] PeterYim: MichaelDenny: ideas and feedback solicited on the survey design between now and Feb-22    (3N1S)
	[10:20] MichaelDenny: re: all commenters, thank you -- the current set of software capabilities 
	under each ontology development phase are concluded by a solicitation to the software provider to 
	add their software capabilities that are not already covered. The objective of this survey is simply 
	to compile an inventory of software resources that help users evaluate or promote ontology quality 
	and fitness. There is no present intent to develop these findings into an ontology of 
	evaluation/fitness factors or capabilities.    (3N1T)
	[10:23] AmandaVizedom: @MichaelDenny: Is there a notion of modularity in the survey, as 
	@MichaelGruninger just discussed I didn't see it, but could have missed)? If not, it seems well 
	worth adding, since it can be important to suitability and there is so much recent and ongoing work 
	on it. Yes?    (3N1U)
	[10:36] MichaelDenny: @AmandaVizedom: Modules are covered under Management phase (as #4), but should 
	probably appear in Build phase as well.    (3N1V)
	[10:05] PeterYim: == MichaelGruninger presenting ... see: the [2-Gruninger] slides    (3N1W)
	[10:05] anonymous morphed into MikeBennett    (3N1X)
	[10:06] SimonSpero: Can you count how many different ontologies you have?    (3N1Y)
	[10:16] JoanneLuciano: @SimonSpero Can you define "different"?    (3N1Z)
	[10:22] SimonSpero: Joanne: no.    (3N20)
	[10:08] anonymous morphed into PaulPope    (3N21)
	[10:18] AlanRector: There is work in Manchester on various similar metrics - contact BijanParsia - 
	bparsia [at] cs.man.ac.uk    (3N22)
	[10:21] PeterYim: @AlanRector - thank you, Alan, I believe BijanParsia is being contacted and 
	invited to present at another track session ... right, LeoObrst / SteveRay (Track-A)?    (3N23)
	[10:18] JoanneLuciano: @PeterYim (and other organizers) - do we have any slots in the summit for 
	discussion or any time in the face-to-face set aside for "discussion"    (3N24)
	[10:23] PeterYim: @JoanneLuciano - we will have "discussion" time at the end of each and every 
	virtual session. I am sure there will be plenty of discussion time during the face-to-face Symposium 
	too (although MikeDean & RamSriram, our Symposium co-chairs will be in the best position to provide 
	specifics ... probably closer to the time.)    (3N25)
	[10:27] AmandaVizedom: @MichaelGruninger: Can you say more about "Intended Models" as used in the 
	COLORE sense, and how they might relate to things found "in the wild" - that is, in non-research 
	projects?    (3N26)
	[10:38] MichaelGruninger: @AmandaVizedom: Suppose you are looking for an organization ontology with 
	relations such as supervises, authorizes, manages. You could use an org chart from your company as 
	an example of your intended semantics of the relations. We would translate such a chart into a set 
	of relations with their extensions (i.e. a model) and then search the repository.    (3N27)
	[10:31] SimonSpero: @MichaelGruninger: is there anyway to deal with conservative/definitional 
	extensions without making CLIF's quantification of predicates unhelpful in the cases where they be 
	most useful? Would making CL sorted be enough?    (3N28)
	[10:35] MichaelGruninger: @SimonSpero: all of the work we have done so far has used essentially 
	good-old fashioned FOL, and really hasn't exercised the full quantification of predicates in CL. We 
	translated CLIF to Prover9 and do the theorem proving there. Once there is a full CL theorem prover, 
	we can use it.    (3N29)
	[10:31] AlanRector: On modules - there is an OWL module extractor as a web app on the 
	http://owl.cs.manchester.ac.uk/ site. It is also available by request to download but new features 
	are being added regularly. [above link updated from original post, thanks to prompt from SimonSpero 
	and ElieAbiLahoud.]    (3N2A)
	[10:32] FabianNeuhaus: @Till. Michael mentioned "Mossakowski, T., Codescu, M., Kutz, O., Lange, C., 
	and Gruninger, M. (2013) Proof Support for Common Logic." in his presentation. Is the paper available?    (3N2B)
	[10:43] FabianNeuhaus: @Michael: I think we need to scope the notion of "evaluation". Much of your 
	presentation covered an analysis of the relationships between ontology (e.g., conservative 
	extension, non-conservative extension) etc. This is interesting, but this does not tell you anything 
	about the quality of an ontology or a set of ontologies. .    (3N2C)
	[10:47] MichaelGruninger: @Fabian: First of all, one way in which we evaluate the quality of an 
	ontology is wrt intended models, and we use the relationships between the ontologies in the 
	repository to do this verification. In other words, consistency alone is not enough. Second, in past 
	few weeks, people have raised the idea of ontology comparison as being part of evaluation; it does 
	not address quality per se, but it does evaluate two ontologies against each other e.g. what are 
	their similarities and differences?    (3N2D)
	[10:52] FabianNeuhaus: @Michael, second point. Yes, this is why I think we need to scope it, in 
	particular for the sake of the communique. If we use the word "evaluation" to mean "analyze" ontologies 
	or their use in systems under any conceivable aspect, including their structure, community use, 
	and intellectual property rights " then we will have trouble to get to a reasonable communique.    (3N2E)
	[10:55] FabianNeuhaus: @Michael, first point. I understood the point, and agree that one very good 
	way to evaluate ontologies is to use intended models.    (3N2F)
	[10:55] WillBurns: Intellectual property rights as an issue makes the assumption that the data 
	itself has a context which uniquely identifies it and grants it the ability to be copyrighted. In 
	this manner, I suppose we're still addressing the ontology aspect as a 1:1 storage and retrieval 
	methodology whereby the files themselves are married to context. Unfortunately that doesn't *quite* 
	work out in a situation where the volume of data is no longer inherent with context.    (3N2G)
	[10:57] SimonSpero: @WillBurns: In US, Australia, and possibly Canada, "mere data" cannot be 
	copyrighted (beyond selection, coordination, and arrangement).    (3N2H)
	[10:58] WillBurns: @SimonSpero Correct. Specific and unique arrangements can be copyrighted, but the 
	question becomes what happens when we remove all three up front and still manage to store it all?    (3N2I)
	[11:02] MichaelDenny: @FabianNeuhaus: By and large I was being driven by the full range of 
	evaluation factors mentioned in the three tracks which includes, for example, "intellectual property 
	rights". Such usage requirements that go into a user's choice to adopt particular ontologies as 
	components may determine the overall quality or fitness of their ontology product. That said, I also 
	hesitated to include evaluation of this factor as a candidate software capability. I chose to 
	include it to be faithful to the tracks' identifications of factors.    (3N2J)
	[10:27] PeterYim: == JeanneHolm presenting ... see: the [3-Holm] slides    (3N2K)
	[10:43] SteveRay: Looks like the open government github platform shouldn't have a "t". Instead, it 
	is https://github.com/opengovplatform 
	[this is in reference to JeanneHolm's slide#15, which is now corrected. Thanks, Steve.]    (3N2L)
	[10:39] JoanneLuciano: Would like to bring everyone's attention to some tools from our lab at RPI 
	(Tetherless World Constellation) that to take gov't data and turn it to RDF - there's also data 
	quality tools available. This is the RPI data.gov site: http://data-gov.tw.rpi.edu/wiki Linked Data 
	Quality Reports: Tim Lebo https://github.com/timrdf/DataFAQs/wiki    (3N2M)
	[10:48] JoanneLuciano: @JeanneHolm Semantic Ecology and Environmental Portal uses gov't data: 
	http://tw.rpi.edu/web/project/SemantEco    (3N2N)
	[10:49] JeanneHolm: Thanks Joanne! Great pointer.    (3N2O)
	[11:13] ScottHills: @JeanneHolm, on your slide 16, can you offer a link to more info about 
	ADMS-based metadata?    (3N2P)
	[11:15] JeanneHolm: @ScottHills You can find out more about ADMS (a European Union initiative) go to 
	http://joinup.ec.europa.eu/asset/adms/home    (3N2Q)
	[11:16] ScottHills: @JeanneHolm, thanks.    (3N2R)
	[10:51] PeterYim: == GavinMatthews presenting ... see: the [4-Matthews] slides    (3N2S)
	[10:58] DougFoxvog: @Gavin: you show a "RT:" (related term?) link. Does the ontology specify the 
	type of relation? Or is the number of relations very limited?    (3N2T)
	[10:59] SimonSpero: @doug RT is usually defined as a residual category of all relations that aren't 
	equivalence or hierarchical.    (3N2U)
	[11:00] TillMossakowski: slide 6: what does "BT" mean? subconcept? why is "woman musician" a 
	subconcept of "famous musician"?    (3N2V)
	[11:01] SimonSpero: @Till: That looks like an incorrect BT    (3N2W)
	[11:02] SimonSpero: @Till: ?N BT ?B means that everything about ?N is also about ?B    (3N2X)
	[11:03] DougFoxvog: The ontology on slide 6 shows both "woman musician" and "musical performer" as 
	subtypes of "famous musician". Yet "famous musical performer" is a subtype of "musical performer". 
	The class "famous musician" probably is not mis-named since it is a subtype of "celebrity". There 
	seems to be a problem here.    (3N2Y)
	[11:04] SimonSpero: @Till: Diagnostic sentence frame - " It's about a woman musician but it's not 
	about a famous musician" should be unacceptable    (3N2Z)
	[11:15] GavinMatthews: @TillMossakowski: BT (Broader Term) means sub-collection/sub-type. I'm not 
	claiming our ontology is perfect. I wanted to show how we detected errors. :)    (3N30)
	[11:16] SimonSpero: @Gavin: Is it restricted to ~genls?    (3N31)
	[11:18] SimonSpero: @Gavin: In controlled vocabularies, BT can apply to genls, isa, or intentionally 
	necessary parts (where the absence of a part requires explanation)    (3N32)
	[11:19] SimonSpero: @Gavin: there are BTG, BTP, and BTI which are spec preds of BT    (3N33)
	[11:20] SimonSpero: @Gavin: but undifferentiated broader does not allow you to infer a genls    (3N34)
	[11:21] GavinMatthews: @SimonSpero: We use BT to mean roughly "genls". We distinguish ISA meaning 
	isa. For practical reasons, we permit a certain underspecification.    (3N35)
	[11:24] AmandaVizedom: During last week's discussion of development methodology, we really talked 
	very little about how evaluation can guide development. The evaluation Gavin discusses is used by 
	ontology developers strongly; we might want to talk about that.    (3N36)
	[11:05] MatthewWest: Sorry need to drop off now. Considerable food for thought here.    (3N37)
	[11:05] SimonSpero: See: http://dc2008.de/wp-content/uploads/2008/10/04_spero_poster.pdf    (3N38)
	[11:06] SimonSpero: What's needed is D. Allan Cruse control..    (3N39)
	[11:08] DougFoxvog: @Simon: yes, "related term" might be all that is needed for generic web 
	searching. But it would be useful if a search for owners of a sports team would yield different 
	links than for members of the team, or for supporters of the team.    (3N3A)
	[11:09] SimonSpero: @doug: Yes - RT can be specialized (he described a few, I think).    (3N3B)
	[11:24] GavinMatthews: @DougFoxvog: Yes, "RT" means related term. It is our vaguest relation, and we 
	constantly work to specialise it. Relations like "plays role in", "located in", and "has theme" are 
	considered to be sub-relations of RT, but in practice all of those horizontal links have 
	approximately the same effect on disambiguation.    (3N3C)
	[11:30] SimonSpero: @Gavin: Underspecification is necessary - there have been attempts to create 
	more specific coverings of associative relationships ( RT), but they all ran in to the problem that 
	they were incomplete, and not useful for IR purposes    (3N3D)
	[11:09] AmandaVizedom: I would call everyone's attention to Gavin's slide 9. This level of testing 
	is something almost no one imagines applying to ontologies, even in contexts where the rest of the 
	software environment is so tested. Yet it's entirely possible, and once established, extremely valuable.    (3N3E)
	[11:35] AmandaVizedom: Thanks Gavin!    (3N3F)
	[11:11] PeterYim: == Q&A and Open Discussion ...    (3N3G)
	[11:13] MichaelGruninger: @Fabian: A similar comment can be made about ontology metrics i.e. what do 
	they have to do with ontology quality? I get more insight from knowing that an ontology I have is 
	inconsistent with another ontology than I get from knowing that there are 40 axioms with 5 relations, 
	12 classes with 3 siblings.    (3N3H)
	[11:14] TillMossakowski: @Michael: what does "inconsistent with" mean? The union being inconsistent?    (3N3I)
	[11:14] FabianNeuhaus: @Michael. I would (and have) made the same comment about these metrics.    (3N3J)
	[11:20] MichaelGruninger: @Till: Yes, I meant that two ontologies are mutually inconsistent. For 
	example, one ontology entails that time is discrete and the other ontology entails that time is dense.    (3N3K)
	[11:26] JoanneLuciano: One simple way evaluation can be used to guide development is with the use of 
	reasoners. Whenever we add a class or an assertion, we run the reasoner and check if we get what we 
	expect. Of course this is by hand, and scaling this is another story.    (3N3L)
	[11:27] JoaoPauloAlmeida: yes    (3N3M)
	[11:32] SteveRay: Must run. Thanks.    (3N3N)
	[11:33] AlanRector: On tracking environments: We have used Redmine successfully and found it easier 
	than Bugzilla because of the good Wiki and other facilities. So many of the issues that come up are 
	more complex issues than simple "bugs".    (3N3O)
	[11:35] SimonSpero: @AlanRector: bug report - running the owltoskos web tool on the skos ontology crashed.    (3N3P)
	[11:38] AlanRector: @SimonSpero - please email Bijan Parsia - bparsia [at] cs.man.ac.uk - and Sean 
	Becchofer - sean.bechhofer [at] manchester.ac.uk - they will redirect it to whomever is dealing with 
	it now.    (3N3Q)
	[11:35] JoanneLuciano: Please email me if you'd like to collaborate on a generalized ontology 
	evaluation and development framework. jluciano [at] rpi.edu Thanks everyone    (3N3R)
	[11:37] PeterYim: join us again, same time next week, for OntologySummit2013 session-06: "Ontology 
	Summit 2013: Synthesis-I" - Co-chairs: MichaelGruninger & MatthewWest - 
	http://ontolog.cim3.net/cgi-bin/wiki.pl?ConferenceCall_2013_02_21    (3N3S)
	[11:38] GavinMatthews: Thanks very much everyone. I've found it very interesting.    (3N3T)
	[11:40] PeterYim: -- session ended: 11:39 am PST --    (3N3U)
 -- end of in-session chat-transcript --    (3MHX)

Additional Resources:    (3MI4)

For the record ...    (3MIB)

How To Join (while the session is in progress)    (3MIC)

Conference Call Details    (3MFN)

Attendees    (3MGK)