OntologySummit2013: Panel Session-12 - Thu 2013-04-04    (3OR2)

Summit Theme: "Ontology Evaluation Across the Ontology Lifecycle"    (3OR3)

Session Topic: Ontology Summit 2013: Synthesis-II    (3Q4F)

Summit General Co-chairs & session Co-chairs: - intro slides    (3Q4G)

Panelists / Briefings:    (3Q4I)

Archives:    (3Q4O)

Abstract:    (3Q6J)

OntologySummit2013 Session-12: "Synthesis-II" - intro slides    (3Q6K)

This is our 8th Ontology Summit, a joint initiative by NIST, Ontolog, NCOR, NCBO, IAOA & NCO_NITRD with the support of our co-sponsors. The theme adopted for this Ontology Summit is: "Ontology Evaluation Across the Ontology Lifecycle."    (3Q6L)

Currently, there is no agreed methodology for development of ontologies, and there are no universally agreed metrics for ontology evaluation. At the same time, everybody agrees that there are a lot of badly engineered ontologies out there, thus people use -- at least implicitly -- some criteria for the evaluation of ontologies.    (3Q6M)

During this OntologySummit, we seek to identify best practices for ontology development and evaluation. We will consider the entire lifecycle of an ontology -- from requirements gathering and analysis, through to design and implementation. In this endeavor, the Summit will seek collaboration with the software engineering and knowledge acquisition communities. Research in these fields has led to several mature models for the software lifecycle and the design of knowledge-based systems, and we expect that fruitful interaction among all participants will lead to a consensus for a methodology within ontological engineering. Following earlier Ontology Summit practice, the synthesized results of this season's discourse will be published as a Communique.    (3Q6N)

We have now completed the virtual sessions of the Summit that were dedicated to presentations of technical content.Each of the four tracks have hosted very exciting presentations that address the key Summit themes -- Intrinsic Aspects of Ontology Evaluation, Extrinsic Aspects of Ontology Evaluation, Building Ontologies to Meet Evaluation Criteria, and Software Environments for Evaluating Ontologies.    (3Q7V)

In today's session, we will focus on revisiting the synthesis of all of these ideas as input into the initial draft of the Summit Communiqué.    (3Q6P)

The Synthesis II session will be framed by the Communique outline. Track champions will provide discussion questions that represent the points of synthesis they need to address but feel that they don't have enough input to synthesize.    (3Q7W)

More details about this OntologySummit is available at: OntologySummit2013 (homepage for this summit)    (3Q6Q)

Agenda:    (3Q6R)

OntologySummit2013 - Panel Session-12 - Synthesis-II    (3Q6S)

Proceedings:    (3Q73)

Please refer to the above    (3Q74)

IM Chat Transcript captured during the session:    (3Q75)

 see raw transcript here.    (3Q76)
 (for better clarity, the version below is a re-organized and lightly edited chat-transcript.)
 Participants are welcome to make light edits to their own contributions as they see fit.    (3Q77)
 -- begin in-session chat-transcript --    (3Q78)
	------
	Chat transcript from room: summit_20130404
	2013-04-04 GMT-08:00 [PDT]
	------    (3QCZ)
	[9:16] PeterYim: Welcome to the    (3QD0)
	 = OntologySummit2013: Virtual Panel Session-12 - Thu 2013-04-04 =    (3QD1)
	Summit Theme: Ontology Evaluation Across the Ontology Lifecycle    (3QD2)
	Session Topic: Ontology Summit 2013: Synthesis-II    (3QD3)
	* Session Co-chairs 
	 - Professor MichaelGruninger (U of Toronto, Canada) and Dr. MatthewWest (Information Junction, UK)    (3QD4)
	Agenda:    (3QD5)
	* Professor MichaelGruninger (U of Toronto, Canada)
	 - "Thoughts on Ontology Summit 2013 and session intro"    (3QD6)
	* Dr. MatthewWest (Information Junction, UK) - "Reflections on Ontology Summit 2013"    (3QD7)
	* Dr. LeoObrst (MITRE) & Dr. SteveRay (CMU)
	 - "Track-A: Intrinsic Aspects of Ontology Evaluation - Synthesis-2"    (3QD8)
	* Mr. TerryLongstreth (Ind. Consultant) & Dr. ToddSchneider (Raytheon)
	 - "Track-B: Extrinsic Aspects of Ontology Evaluation - Synthesis-2"    (3QD9)
	* Dr. MatthewWest (Information Junction) & Mr. MikeBennett (EDM Council; Hypercube)
	 - "Track-C: Building Ontologies to Meet Evaluation Criteria - Synthesis-2"    (3QDA)
	* Dr. MichaelDenny (MITRE) & Mr. PeterYim (Ontolog; CIM3)
	 - "Track-D: Software Environments for Evaluating Ontologies - Synthesis-2"    (3QDB)
	* Dr. AmandaVizedom (Ind. Consultant) & Dr. FabianNeuhaus (NIST), moderators
	 - Open Discussion on how the synthesized ideas may be represented in the Communique draft    (3QDC)
	Logistics:    (3QDD)
	* Refer to details on session page at: http://ontolog.cim3.net/cgi-bin/wiki.pl?ConferenceCall_2013_04_04    (3QDE)
	* (if you haven't already done so) please click on "settings" (top center) and morph from "anonymous" to your RealName (in WikiWord format)    (3QDF)
	* Mute control: *7 to un-mute ... *6 to mute    (3QDG)
	* Can't find Skype Dial pad?
	** for Windows Skype users: it may be under the "Call" dropdown menu as "Show Dial pad"
	** for Linux Skype users: please note that the dial-pad is only available on v4.1 (or later or the earlier Skype versions 2.x,)
	   if the dialpad button is not shown in the call window you need to press the "d" hotkey to enable it.    (3QDH)
	* Note: ... it has come to our attention that our conference bridge provider is running into some 
	problems with the "joinconference" skype connections. In case anyone gets in trouble, please try to 
	call the phone numbers instead (e.g. from your phone, skype-out, google-voice, etc.)    (3QDI)
	Attendees: MichaelGruninger (co-chair), MatthewWest (co-chair), AliHashemi, AmandaVizedom, 
	BobbinTeegarden, CarmenChui, DougFoxvog, FabianNeuhaus, FranLightsom, FrancescaQuattri, FrankLoebe, 
	JackRing, JoelBender, JulienCorman, KenBaclawski, LamarHenderson, LeoObrst, MarcelaVegetti, 
	MaryPanahiazar, MeganKatsumi, MichaelDenny, MikeRiben, PavithraKenjige, PeterYim (scribe), SteveRay, 
	TerryLongstreth, TillMossakowski, ToddSchneider,    (3QDJ)
	 == Proceedings: ==    (3QDK)
	[9:23] anonymous morphed into CarmenChui    (3QDL)
	[9:25] anonymous1 morphed into MichaelDenny    (3QDM)
	[9:25] anonymous morphed into FrancescaQuattri    (3QDN)
	[9:30] PeterYim: @FrancescaQuattri - did you just connect to the call? (that connection was 
	injecting a lot of noise into the line; you'll need to stay on mute when not speaking)    (3QDO)
	[9:31] FrancescaQuattri: yup    (3QDP)
	[9:31] FrancescaQuattri: Hi Everybody    (3QDQ)
	[9:32] anonymous morphed into MaryPanahiazar    (3QDR)
	[9:33] anonymous1 morphed into JulienCorman    (3QDS)
	[9:34] anonymous morphed into BobbinTeegarden    (3QDT)
	[9:34] JoelBender: @Peter - online with Skype - no microphone    (3QDU)
	[9:33] PeterYim: Hello mary panahiazar, Welcome! [ ... send me your email so you can get subscribed 
	to the lists and participate in the async discussion too.]    (3QDV)
	[9:34] MaryPanahiazar: mary [at] knoesis.org    (3QDW)
	[9:35] ToddSchneider: All, I have to leave at 14:00 EDT.    (3QDX)
	[9:36] PeterYim: == MichaelGruninger opens the session ... see: the [ 0-Gruninger ] slides    (3QDY)
	[9:37] List of members: AliHashemi, AmandaVizedom, BobbinTeegarden, CarmenChui, DougFoxvog, 
	FabianNeuhaus, FrancescaQuattri, FrankLoebe, FranLightsom, JoelBender, JulienCorman, KenBaclawski, 
	MaryPanahiazar, MatthewWest, MeganKatsumi, MichaelDenny, MichaelGruninger, PeterYim, SteveRay, 
	TerryLongstreth, ToddSchneider, vnc2    (3QDZ)
	[9:46] SteveRay: With respect to conditions for ontology evaluation, we can talk about necessary 
	conditions for evaluation, and possibly sufficient conditions for evaluation, with respect to 
	various stages of development.    (3QE0)
	[9:43] MichaelGruninger: Outcome hackathon HC05 
	http://ontolog.cim3.net/file/work/OntologySummit2013/Hackathon-Clinics/HC-05_Ontology-of-OntologyEvaluation/wip/HC-05_doc-snapshot_at-end-day-20130331/    (3QE1)
	[9:46] AmandaVizedom: Note about HC-05 outputs: This is snapshot of work at the end of the weekend 
	sessions. Results are dispersed across a number of text and graphic files. Currently, several of us 
	are working on consolidating the conceptual model in both graphical and English text forms, and 
	making sure that we, as a group, agree that this captures what we developed. We are also drafting 
	formal ontologies based on this, in OWL and Common Logic, but all should be considered first drafts, 
	and current push is on the consolidated concept model.    (3QE2)
	[9:55] PeterYim: @Amanda, Ali, et al. - at the OntoIOp working group meeting yesterday, 
	TillMossakowski and I were kicking around the idea of hacking up a demo (for the 
	OntologySummit2013_Symposium), to evaluate two manually developed versions of the "Ontology of Ontology 
	Evaluation" (a la HC-05 - in OWL and CLIF), and two machine-translated versions of those Ontologies 
	(of Onto Eval) with Hets / DOL / OntoIOp / Ontohub (OWL->CLIF; CLIF->OWL) ... and run them through 
	some of the tools featured during this summit ... it'll be fun!    (3QE3)
	[10:00] AmandaVizedom: @Peter: Excellent! I've been a bit dissatisfied that even with our follow-on 
	commitments to create the formal ontologies, we haven't had a specific plan for evaluating them. And 
	that's no good, from the practicing what we preach perspective. So, in addition to the fun of it, I 
	think that is an excellent idea!    (3QE4)
	[9:47] PeterYim: == MatthewWest presenting ... see: the [ 1-West ] slides    (3QE5)
	[9:51] SteveRay: Interesting: Decision taking (UK) = Decision making (USA)    (3QE6)
	[9:56] anonymous morphed into LamarHenderson    (3QE7)
	[9:58] AmandaVizedom: Cost reduction benefits, and sponsor's ROI in general, were brought into our 
	HC-05 discussions this weekend, advocated especially by BobSmith. Figuring out how these fit into 
	the high-level evaluation has been a challenge. MatthewWest's comments related to his slide 3 
	suggests to me that we began to model requirements and their large dependence on usage, and we began 
	to model aspects of usage, and we began to model purpose as part of that, but under purpose we 
	focused on delivered functionality. Matthews slide 3 highlights delivered benefits, at a higher 
	level than specific functionalities. That, I think, we need to add explicitly.    (3QE8)
	[9:59] PeterYim: == SteveRay presenting ... see: the [ A-Obrst-Ray ] slides    (3QE9)
	[10:05] DougFoxvog: (in response to discussion of Slide 2 of Track A) Class vs. instance distinction 
	being questionable arises if the ontology makes the two disjoint. If classes may be used as 
	arguments to predicates (and metaclasses are allowed), then one need not make the narrowest classes 
	into instances of their superclasses.    (3QEA)
	[10:15] PeterYim: == ToddSchneider presenting ... see: the [ B-Schneider-Longstreth ] slides    (3QEB)
	[10:15] TerryLongstreth: (ref. ToddSchneider's remark that he will present, as TerryLongstreth is 
	having trouble talking) I'm listening, but as Todd says, having trouble with verbal communication    (3QEC)
	[10:17] SteveRay: Disagree with Terry in calling OOPS! a blackbox evaluation. It is specifically 
	examining the contents of the ontology - opening up the box and looking for structural errors.    (3QED)
	[10:18] TerryLongstreth: That was Todd, but I think he was just illustrating the ambiguity of the 
	dichotomy    (3QEE)
	[10:18] MatthewWest: @Ray: I would expect intrinsic properties to become important (or not) in 
	supporting higher level extrinsic requirements. So the key is to understand the way higher level 
	requirements are supported by requirements for generally lower level, intrinsic properties.    (3QEF)
	[10:19] AmandaVizedom: @Matthew +1 (independently of Steve's comments or OOPS!).    (3QEG)
	[10:20] SteveRay: @Matthew: I agree. Intrinsic evaluation alone has no value unless related to the 
	ultimate system performance.    (3QEH)
	[10:20] DougFoxvog: I agree with Steve. OOPS! ignores the *meaning* of the terms, but has access to 
	all the statements in the ontology. Ignoring the meaning seems to be what Todd meant by "black box".    (3QEI)
	[10:22] SteveRay: @Doug: You may be right in how Todd (sorry Terry, got the names swapped) intended 
	to use the term black box, but that is an odd use of the term, somewhat opposite to what at least I 
	understand it to mean.    (3QEJ)
	[10:21] DougFoxvog: @Matthew, @Amanda: +1    (3QEK)
	[10:21] MichaelGruninger: @DougFoxvog: What do you mean by "ignoring the meaning"? The "meaning" of 
	a term should be equivalent to the possible interpretations of the axioms    (3QEL)
	[10:23] DougFoxvog: The "meaning" of the term is defined for humans and humans use that meaning for 
	labeling (e.g., cells on a slide, info on medical records, etc.)    (3QEM)
	[10:25] DougFoxvog: @Michael: I agree that the meaning of an ontology in a vacuum is just the 
	possible interpretations of the axioms. However, ontologies are (hopefully) used in conjunction with 
	other systems, and so their mappings to those systems affects the meaning of the terms.    (3QEN)
	[10:28] MichaelGruninger: @DougFoxvog: In the work with MeganKatsumi, the intended meanings of terms 
	are requirements that are formalized as intended models. We can then evaluate the ontology (using 
	the axioms alone) to determine whether or not it meets those requirements i.e. whether or no there 
	are intended models. When ontologies are used together, the intended models need to be in common.    (3QEO)
	[10:25] AmandaVizedom: @Todd: While discussing slide 3, you said that the evaluation has a context, 
	and that when you know that context, then you can rank the results of your evaluation (metrics, 
	etc). This sounds to me like a different framing, but in principle equivalent to a different process 
	characterization that we have discussed. In this other characterization, The context comes first -- 
	specifying the intended usage, gathering requirements. From this, evaluation criteria are identified 
	that are relevant to answering whether these specific, context-driven requirements are satisfied, 
	and evaluation is conducted over those criteria. Do you agree that both processes emphasize the 
	contextuality of evaluation relevance equivalently?    (3QEP)
	[10:28] DougFoxvog: @Amanda: Should we expect the contexts to be defined (as you said they must be) 
	using an ontology? I.e., are the context definitions to be stated in a formal logic using terms 
	defined in an ontology?    (3QEQ)
	[10:34] AmandaVizedom: @doug, yes, though here I am using context as I think Todd meant it, not in 
	all the possible ways I might otherwise be found using it. ;-) In the HC-05 model, we've been so far 
	following along with the Ontology Usage characterization seeds laid down in the 2011 summit. That 
	is, the formalized characterization of context consists partially in the explicit capture of various 
	aspects of the usage (including things like application type, users, and so on), not yet nearly 
	exhaustively captured. Priority is on such characteristics as we come to understand that they make a 
	difference to what ontology features are needed.    (3QER)
	[10:26] SteveRay: @Michael: I'd be interested in your thoughts on the axioms when one is presented 
	with, say, an OWL file that contains only sub/superclass relations and some allValuesFrom or 
	someValuesFrom relations. In other words, no explicit axioms at all.    (3QES)
	[10:32] MichaelGruninger: @Steve: I would say that subclass relations are still axioms. Of course, 
	if these are all you have, then there will most likely be many possible interpretations of the 
	ontology that do not correspond to the intended meanings. A great example of this is the 
	relationship between OWL-S and SWSO. In cases such as this, I wonder what the requirements for the 
	ontology are considered to be.    (3QET)
	[10:27] PeterYim: @Todd - (re. your remark during slide#7) I somewhat disagree that "testers are not 
	familiar with ontologies" ... if we look at (and we should) test designers as among the "testers" 
	(that's the group that's meaningful, we should not be talking about the test operators), then they 
	simply do not qualify for the job if they are not familiar with ontologies    (3QEU)
	[10:34] ToddSchneider: Peter, I qualified 'tester' to be in the context of system integration 
	testing (i.e., the end of the development phases and prior to deployment).    (3QEV)
	[10:37] PeterYim: @Todd - fair!    (3QEW)
	[10:29] PeterYim: == MatthewWest presenting ... see: the [ C-West-Bennett ] slides    (3QEX)
	[10:33] DougFoxvog: Slide 3: "The physical level would be an encoding in a formal language" such as 
	OWL. This is an interesting definition of "physical". It would be nice for the slide to be edited to 
	clarify this meaning. I might call this the "code" level.    (3QEY)
	[10:38] AmandaVizedom: @Matthew - during HC-05, we found your Conceptual / Logical / Physical stages, 
	following DB usage someone, to make the most sense when mapped thusly: Conceptual: human-centric 
	capture in one or more artifacts, could be textual, graphical, combined, rigorous but not formal. 
	Logical: expressed in a formal ontology language. Physical: expressed in a serialization of such a 
	language. Is this compatible with your thinking?    (3QEZ)
	[10:47] MatthewWest: @Amanda: Possibly. In truth there are variations in interpretation of the 
	levels in the database world. Certainly the physical level is what is in the system running queries. 
	The logical level is an abstraction of that that is not implementation environment specific. I would 
	probably want to say that you would not have committed to FOL or DL yet, but we could debate that 
	(maybe another level?)    (3QF0)
	[10:44] ToddSchneider: Matthew, Instead of 'quality', would 'value' be a notion that better conveys 
	our intent?    (3QF1)
	[10:35] PeterYim: == MikeDenny presenting ... see: the [ D-Denny-Yim ] slides    (3QF2)
	[10:39] LeoObrst: Finally joining. Sorry I'm late.    (3QF3)
	[10:39] PeterYim: glad you made it, Leo!    (3QF4)
	[10:46] TerryLongstreth: Track D makes a good point that much of our work has seemed to presume a 
	Waterfall model of development. We didn't explicitly talk about it but the Track B concerns with 
	dynamics are probably best illustrated in current practice by environments by dynamic injection of 
	new or unanticipated requirements as happens in agile development situations.    (3QF5)
	[10:48] DougFoxvog: There have been several mentions that symmetric, reflexive, and transitive 
	predicates should have the same domain and range. This is true for symmetric predicates, but for 
	transitive predicates, the requirement should be that the range is a subclass of the domain. For 
	reflexive predicates, it really depends upon one's definition of "reflexive" -- does it mean 
	 (forAll (X P P_Range P_Domain) (implies (and (isa P BinaryPredicate) (range P P_Range) (domain P P_Domain) (isa X P_Range) (isa X P_Domain)) (P X X))) 
	or 
	 (forAll (X P P_Range P_Domain) (implies (and (isa P BinaryPredicate) (range P P_Range) (domain P P_Domain) (isa X (ClassUnionFunction P_Range P_Domain)) (P X X))) 
	In the second case, the domain & range must be the same. In the first, they should just not be disjoint.    (3QF6)
	[10:58] SteveRay: @MichaelDenny: Indeed, some of us are trying to link ontology evaluation to 
	traditional modeling tools. I and my team convert Enterprise Architect files into OWL, and then 
	apply various evaluation queries against them using SPARQL. One example of output can be found at 
	http://fsgim.sv.cmu.edu    (3QF7)
	[11:03] MichaelDenny: @SteveRay Very interesting and glad to see it. I will take a look.    (3QF8)
	[11:00] ToddSchneider: Have to go.    (3QF9)
	[11:01] PeterYim: == Q&A and Open Discussion on how all of these ideas should be captured into the 
	OntologySummit2013_Communique ... moderated by FabianNeuhaus and AmandaVizedom    (3QFA)
	[11:01] PeterYim: please refer to communique outline at: 
	http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013_Communique/Draft#nid3O16    (3QFB)
	[11:04] ToddSchneider: Amanda, Fabian, One suggestion before I really leave, I'd suggest dropping 
	the in/extrinsic distinction and replace it with the lifecycle phase. It seems a better criteria for 
	making evaluations distinctions.    (3QFC)
	[11:06] SteveRay: @Todd: Not sure I agree with this. Lifecycle has to do with WHEN, or at which 
	phase, does one evaluate. The intrinsic/extrinsic distinction relates to WHAT one is evaluating.    (3QFD)
	[11:10] TerryLongstreth: Lifecycle phases may also have multiple contexts: to the developer, the 
	lifecycle phase labeled development is (one of) his operational swimming pools. He may touch more 
	than one ontology if for example, the development environment is driven by an ontology (Rational 
	products have that flavor, if not directly employing the term)    (3QFE)
	[11:04] PeterYim: +1 to, at least, the first half of Todd's suggestion. I think the 
	intrinsic/extrinsic distinction served a useful purpose to help us frame the discourse, but 
	introducing this "new terminology" is as confusing as not introducing it at all    (3QFF)
	[11:09] MeganKatsumi: @SteveRay, @Todd: I agree that the in/extrinsic distinction is confusing, but 
	I also think that Steve has a point about the proposed using of the lifecycle phase. Might another 
	useful distinction be the idea of functional/non-functional requirements/attributes?    (3QFG)
	[11:12] MichaelDenny: @MeganKatsumi I have suggested "model quality" vs "domain fidelity" vs 
	application fitness.    (3QFH)
	[11:12] SteveRay: @Michael: I like your partitioning.    (3QFI)
	[11:12] MatthewWest: I also agree that intrinsic/extrinsic has not been helpful. However, I don't 
	think it matters very much. It gave us a way to start, and we can move on from that.    (3QFJ)
	[11:12] AmandaVizedom: As Fabian is saying on the conversation now, we do not plan on using the 
	intrinsic/extrinsic distinction an organizer of the Communique. See outline.    (3QFK)
	[11:14] PeterYim: +1 to what FabianNeuhaus just said about how he and AmandaVizedom are planning to 
	lay out the communique    (3QFL)
	[11:06] AmandaVizedom: This is also of great potential use to Enterprise Architecture and Business 
	Process Management practices themselves, and the development of semantic IT to better support them. 
	Enterprise semantic tech projects are often based in information sharing needs related to business 
	processes. In best cases, that basis is somewhat clear from documentation of business process and EA 
	environment from just such tools. But these tools stop at the level of the input, output, or sharing 
	of information bearing objects (reports, data sets, messages). They don't drill down into the 
	information *contents*. That is precisely where the ontology coverage needs and scoping of the 
	semantic projects picks up, and it is much more effectively captured and conveyed within a context 
	of continuity with those EA/BP models.    (3QFM)
	[11:06] TerryLongstreth: @Fabian - (in reference to Fabian's verbal remarks on how Track-A and 
	Track-B focused their discourse, and the gap) Track B wasn't so concerned with the physical level as 
	the behavioral consequences to the system of having ontology or an ontology within it.    (3QFN)
	[11:10] FabianNeuhaus: @Terry - yes, that's what I meant, I did not put it very elegantly. My point 
	was that there are some aspects of ontology evaluation/quality that was not covered by either track, 
	should be covered.    (3QFO)
	[11:06] LeoObrst: @MichaelDenny: (ref. slide#5 "That which we call a rose by any other name would 
	smell as sweet.") yes, I call it "a label does not wear its semantics on its sleeve", which a lot of 
	XML and database folks sometimes think, e.g., if a label is named "Person", well of course I know 
	what it means! This is also encouraged by very long camelCase concept names like 
	PersonsWhoWieldHammers, where the label seems to be the composition of the semantics of natural 
	language terms.    (3QFP)
	[11:09] MichaelDenny: @LeoObrst ...or "you can't tell a concept by its cover"    (3QFQ)
	[11:12] anonymous morphed into PavithraKenjige    (3QFR)
	[11:15] JackRing: Seems to me any ontology must be evaluated with respect to domain-specific (usage) 
	and discipline-specific (principles and standards) contexts. Further, an ontology can be evaluated 
	for quality (what it is, what it does and what it knows), parsimony and beauty. I sense confusion 
	about whether ontology serves as framework, praxis, system or what?    (3QFS)
	[11:19] JackRing: Life cycle is a distracting notion. Most all ontologies evolve and morph. It may 
	be better to telk in terms of Usage Scenario.    (3QFT)
	[11:14] MichaelGruninger: Does it make sense to consider specific ontology evaluation tasks, and 
	then specify what the inputs to the tasks are? e.g. is evaluation done with respect to the 
	ontology's axioms alone? Is the ontology evaluated wrt a specific set of requirements?    (3QFU)
	[11:18] MatthewWest: @Michael: You can only evaluate against requirements. If you look at my slide 
	on Properties key to Information Quality, you will find properties at a level that business folk can 
	state their requirements at. But then take consistency. What are the more detailed properties of an 
	ontology that you can measure that tell you about its consistency? how do you transform the 
	requirements at the business level down to this level?    (3QFV)
	[11:20] MichaelGruninger: @MatthewWest: Some of the criteria in Steve and Leo's slides use only the 
	axioms of the ontology    (3QFW)
	[11:18] AmandaVizedom: I will add that I believe that there are many ways of "slicing and dicing" 
	ontology characteristics/ requirements / evaluation criteria. Intrinsic/extrinsic is one (or 
	several, given the various interpretations), as are lifecycle stages, relationship to aspects of 
	usage/ relationship to some aspect of theory, etc.. And different tools and methodologies utilize 
	different such organizations. What's more important is that we understand what the 
	characteristics/criteria/requirements are, and when & why they matter, and how & when they may be 
	evaluated.    (3QFX)
	[11:19] MatthewWest: @Amanda: +1    (3QFY)
	[11:21] MeganKatsumi: @Amanda: +1    (3QFZ)
	[11:22] DougFoxvog: @Amanda: +2. The ontology evaluation ontology should have concepts and relations 
	for all that.    (3QG0)
	[11:20] DougFoxvog: Leo is discussing properties of different ontology aspects relative to life 
	cycle phase. If the specific relations are written down, they could be encoded using the ontology 
	evaluation ontology.    (3QG1)
	[11:19] PeterYim: Registration (either onsite or remote) is now open for the 
	OntologySummit2013_Symposium at NIST - Thu & Fri May 2~3, 2013 (Gaithersburg, Maryland, USA) - see 
	detials at: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013/WorkshopRegistration 
	(registration for onsite attendance is mandatory ... so note the Apr-22 registration deadline!)    (3QG2)
	[11:20] PeterYim: Join in the fun at this weekend's Hackathon-Clinics Activities - see details at: 
	http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2013_Hackathon_Clinics#nid3PG4 ... even if you 
	don't plan to hang around all day, you might be interested to participate at the "open webcast 
	segment" of the two projects being featured this Saturday (Apr-6)    (3QG3)
	[11:20] PeterYim: Again, solicitation to software environment stewards and tool developers to 
	respond to the OntologySummit2013 Software Survey - goto: 
	http://ontolog-02.cim3.net/wiki/OntologySummit2013_Survey ... enter name of your tool, and proceed 
	to questionnaire (make sure you complete all phases (questions under all tabs)    (3QG4)
	[11:20] PeterYim: As MichaelGruninger just said, same time next week, for OntologySummit2013 
	session-13: "Communique Draft Review" - Co-chairs: AmandaVizedom & FabianNeuhaus - developing 
	session details at: http://ontolog.cim3.net/cgi-bin/wiki.pl?ConferenceCall_2013_04_11    (3QG5)
	[11:20] MatthewWest: Sorry I have to go now.    (3QG6)
	[11:25] SteveRay: Good session. Thanks!    (3QG7)
	[11:25] PeterYim: great session!    (3QG8)
	[11:25] PeterYim: -- session ended: 11:25 am PDT --    (3QG9)
 -- end of in-session chat-transcript --    (3Q79)

Additional Resources:    (3Q7G)

For the record ...    (3Q7N)

How To Join (while the session is in progress)    (3Q7O)

Attention: Please take special note on the start time of the event, as the US, the EU and some regions are on daylight saving (summer) time on this day (the EU just changed last weekend), while there are other regions that stay on Standard Time all year round!    (3Q4E)

Conference Call Details    (3Q4X)

Attendees    (3Q5U)