ppy/chat-transcript_unedited_20120223a.txt ---------------- Chat transcript from room: summit_20120223 2012-02-23 GMT-08:00 ---------------- [09:20] PeterYim: Welcome to the = OntologySummit2012: Session-07 - Thursday 2012-02-23 = Summit Theme: OntologySummit2012: "Ontology for Big Systems" Track (Cross-Track-A1) Title: Ontology Quality and Large-Scale Systems Session Topic: Implementing Ontology Quality Measures in Big Systems Engineering Session Co-chair: Dr. AmandaVizedom (Wind River) and Mr. MikeBennett (Hypercube, EDM Council) Panelists ... with briefings from: (3543) * Dr. AldoGangemi (ISTC-CNR STLab, Rome, Italy) - "Ontology Evaluation and Pattern-based Design" * Ms. JenniferWilliams (Highfleet, US) - "System requirements and the unobtrusive ontology" * Dr. NicolaGuarino (IAOA; ISTC-CNR LOA, Trento, Italy) - "Ontology quality, ontology design patterns and competency questions" * Dr. AmandaVizedom (Wind River, US) - "Finding or Making Ontology Fit for your Purpose" * along with panelists from other summit tracks (for the discussion) Session page: http://ontolog.cim3.net/cgi-bin/wiki.pl?ConferenceCall_2012_02_23 Mute control: *7 to un-mute ... *6 to mute (please make sure your own phone is not muted as well) Can't find Skype Dial pad? ... it's under the "Call" dropdown menu as "Show Dial pad" . == Proceedings: == [08:42] anonymous morphed into AldoGangemi [09:24] PeterYim: the conference bridge is now open [09:24] anonymous morphed into BobbinTeegarden [09:25] anonymous morphed into TomTinsley [09:26] anonymous morphed into JohnBilmanis [09:27] anonymous morphed into DougFoxvog [09:27] anonymous morphed into AliHashemi [09:29] anonymous morphed into JenniferWilliams [09:29] Nicola Guarino: @Harold: Hi Harold, long time no see... :-) [09:30] Harold Boley: http://www.cs.unb.ca/~boley/ [09:31] JackRing: Q1. A through H where H = evolving the ontology that relates [stakeholder angst- problematic situation] to [intervention system capabilities hypothesis] and evolves to actual intervention system design/architecture model. Q2. Ontology development is the first task of system engineering. Results in basic knowledge knugget that is elaborated thereafter. Q3. Sufficient rigor and accuracy are fundamental. However, interoperability of the languages of the various tribes and viewpoints involved is the real goal. Q4. No. Ontology requirements are typically misleading. Instead, a series of tribal interoperability experiments are conducted to converge in a necessary and sufficient set of ontologies that are fit for purpose. Q5. In-house leading but specialist services are needed. Tools selection not yet pertinent but Top Quadrant assets appear promising. Q6. No ontology exists for field of discourse involving System, System Praxis, Systems Engineering, Fault Detection and Resolution and Model, particularly for autocatalytic kinds of systems. Constructing Concept Maps to clarify/unify extant terminology then will express (no pun intended) these to ontologies, then elaborate with semantic equivalence transformers. Q7. Sense making, integrity, sufficient level of abstraction, cycle time, execution cost, agility, beauty. Q8. Not yet. Ability to measure coherence among micro-ontologies and spectrum of concept labels used by various tribes, e.g., nyms . Q9. Continually. Ambiguities in Peircean relations and robustness of Rosen relationals. Confirmed by tracking design faults back to origins. Note that for Intelligent kinds of systems Fit for Purpose only occurs when no more problems are encountered. Q10. Design faults (multi-human error) cut to less than 50% of typical. Found early so avoids schedule slips, cost overruns and last minute system capability deletions. First clue was the CRC exercises invented by Ward Cunningham and Kent Beck in 1980s. Q11. TBD. Current ontology standards seem too mechanistic and simplistic to handle the web of meanings among N persons, particularly when N > 3. Q12. Configuration Management, particularly change propagation and control. Similar to distributed transaction processing problem except requires three-phase commit, not just two-phase commit. Q13. Unification of humans pet views and ladders of inference especially when premature immaculation has occurred regarding standards and tools. Current tools assist only the linotype operators when we are supposed to be creating poetry. Q14. Are there limits in Extent, Variety (both temporal and semiotic) and Ambiguity, particularly with respect to designer competency? What is the difference between complexness and complexity? Are we obsessing on ontologies while society needs semiotic transformers? Do we suppress variety in fear of combinatorial complexness and presumed computational intractability.? simplistic to handle the web of meanings among N persons, particularly when N > 3. Q12. Configuration Management, particularly change propagation and control. Similar to distributed transaction processing problem except requires three-phase commit, not just two-phase commit. Q13. Unification of humans pet views and ladders of inference especially when premature immaculation has occurred regarding standards and tools. Current tools assist only the linotype operators when we are supposed to be creating poetry. Q14. Are there limits in Extent, Variety (both temporal and semiotic) and Ambiguity, particularly with respect to designer competency? What is the difference between complexness and complexity? Are we obsessing on ontologies while society needs semiotic transformers? Do we suppress variety in fear of combinatorial complexness and presumed computational intractability.? [09:31] Harold Boley: Hi Nicola, long time no see. [09:33] AmandaVizedom: @Jack: Glad you had a chance to look at the discussion questions and have prepared such thoughtful answers! We'll do those Questions in Chat at the end of the session. Can you hold and add your answers to each question as we go through them? [09:34] AmandaVizedom: @Jennifer - Glad you could get into the chat! [09:35] SteveRay: A few people have entered the chat with separate first and last names. Following the convention of LastnameFirstname allows direct integration on our wiki archives, so if you could press the "Settings" button at the top of the chat screen and change your name it would help out a lot. [09:35] SteveRay: Sorry, I meant to say use the convention of FirstnameLastname, not the other way around. [09:35] anonymous morphed into Denise Warzel [09:36] Denise Warzel morphed into DeniseWarzel [09:36] Anatoly Levenchuk morphed into AnatolyLevenchuk [09:37] JackRing morphed into RingJack [09:38] RingJack morphed into JackRing [09:39] anonymous morphed into ValentinaPresutti [09:40] Nicola Guarino morphed into NicolaGuarino [09:41] PeterYim: == AmandaVizedom presenting (session-07 intro) [09:43] PeterYim: == AldoGangemi presenting ... [09:46] AmandaVizedom: currently on slide 6 [09:47] AmandaVizedom: Slides 8-9: responding to Ontology Summit discussion so far [09:52] ToddSchneider: How is 'context' (i.e., the extent to which the other metrics/aspects dependent upon it) represented via any quality measure(s)? [09:58] Gary Berg-Cross morphed into GaryBergCross [10:04] SimonSpero: Test Driven Design (TDD) has shown unequivocal results in all studies (that's where tests are created before the engineered artifact). Pair Programming has not consistently shown improvements in the general software engineering literature [10:06] AmandaVizedom: RaviSharma asks: How many changes do you find when you go from design patterns to implementation? [10:11] AmandaVizedom: == Jennifer Williams presenting ... [10:12] MikeBennett: Hi, sorry I'm late! [10:12] AmandaVizedom: @Mike - Glad you made it. [10:13] ValentinaPresutti: Test-driven can be combined with pair design e.g. XP. Actually XD, the method mentioned by Aldo combined the two. It is test-driven, which has shown a good impact on resulting ontologies and pair designed. On pair design we have still to experiment [10:13] ValentinaPresutti: typo: combined -> combines [10:14] ValentinaPresutti: sorry, my message was meant to respond @SimonSpero [10:15] AldoGangemi: @SimonSpero Actually we have not yet compared test-driven vs pair-design *on ontologies* [10:16] LarryLefkowitz: If ontology quality measurements are tied to the requirements of the application for which it was initially developed, don't we run the risk of "over-fitting" the ontology, thereby possibly reducing the likely re-usability of that ontology? I agree that satisfying the (initial) target application's needs is *necessary*, but I believe not sufficient. [10:17] SimonSpero: @Aldo @Valentina: thanks [10:17] ToddSchneider: Jennifer, to what extent did the need for semantic consistency across components and interfaces impact the design? [10:18] AldoGangemi: @LarryLefkowitz I agree: that's necessary, not sufficient. However, overfitting could be a risk mainly on core or foundational ontologies, which btw have own peculiar requirements. Much less on domain and application-oriented ontologies [10:18] NicolaGuarino: @Larry: I agree very much with your point. Evaluating ontologies wrt one particular application has the risk of forgetting the main reason most ontologies are developed, i.e. facilitating interoperability with *other* applications [10:20] NicolaGuarino: The reference ontology which appears in the case being discussed by Jennifer is indeed *application independent* (sort of...) [10:21] AldoGangemi: @Larry @Nicola global interoperability cannot be provided on *all* ontologies, otherwise we are trapped into maximalistic ontology design efforts all the times [10:22] AldoGangemi: @Nicola patterns are not application-dependent, in general [10:24] SimonSpero: Bit more how (from STIDS 2010 tutorial - http://stids.c4i.gmu.edu/presentations/STIDS_Tutorial_Highfleet.pdf ) [10:26] LarryLefkowitz: @Aldo: Is there a happy medium? Full disclosure: As a Cyc person, my bias is on the global end. My fear is that application-specific (stand-alone) ontologies won't integrate well once they get beyond trivial semantics because so much of the semantics are implicit, which may be acceptable when the scope of application is known (and appropriately restricted). Yes,I agree moving towards "patterns" is a step in the right direction, but until more of the semantics of those patterns (and their domain of applicability) is captured in the ontology, the challenge remains. [10:26] SimonSpero: @Aldo @Nicola : Patterns qua Design patterns should be abstracted [10:28] SimonSpero: @Larry: Cyc 101 had (has) a few sessions that are rather pattern like [10:28] NicolaGuarino: @Jennifer: what is the main thing you ask to your reference ontology, in terms of quality? [10:30] AldoGangemi: @Simon yeah, this is a typical criticism: why making/providing core or even domain design patterns? I agree for logical, architectural, and other types of ODP, but knowledge patterns are a different beast: we don't know in advance how general they should be in order to be "correctly" abstract. My impression is that abstract ones are more useful, but sometimes they do not provide enough beef just because they do not respond to precise-enough requirements [10:30] ValentinaPresutti: @LarryLefkowitz: patterns can (some of them already are) be aligned with more general/abstract ones, and with foundational ontologies so as to ensure interoperability [10:30] ToddSchneider: Jennifer, could say a bit more about the 'config' files? What do they configure? [10:31] LarryLefkowitz: @Simon: I agree with the pattern approach. No one wants to re-invent wheels, so anything that allows/supports reuse is generally progress. Key is understanding when and where that reuse is possible. Textual documentation is a start in that it allows humans to better do that integration. If systems are going to successfully be able to integrate (reuse) these ontologies (or patterns), then they (the ontologies) need to be ever more explicit. [10:33] SimonSpero: @Larry: Absolutely and completely agree on the (incf documentation) point- will hold that till the Q&A session [10:33] BobbinTeegarden: @Jennifer Great talk! Qs: In what form did you deliver the 'ontology' for use; and was this an interative process, or parallel, or...? [10:36] PeterYim: == NicolaGuarino presenting ... [10:37] JackRing: Peter, are you advanciing the slides? [10:37] PeterYim: @10:37 - on slide#5 now [10:39] PeterYim: I should only be advancing on the speaker's prompt ... (otherwise the future async audience will lose sync) ... preesumably, if I don't advance, the speaker will realize he/she has missed the prompt [10:42] AldoGangemi: @Nicola you know we used the blocks world example many times in the past ... it is relevant as soon as you have *explicit* requirements that allow you to exclude non-intended models. Unfortunately, the only way to provide all possible distinctions to exclude those models is to reuse a global, all-encompassing theory of the world, which is suspicious on many grounds. First of all, the cognitive one [10:45] AldoGangemi: @Nicola your notion of interoperability is *maximalistic*, therefore unable to be used in general. On the contrary, current ideas of interoperability are *minimalistic* [10:45] PeterYim: @10:45 - on slide#10 now [10:46] SimonSpero: (I'm feeling like correctness, completeness, and accuracy are a unit or two of polysemy off my idiolect) [10:46] SimonSpero: (Hi MikeBennet) [10:49] AldoGangemi: @Nicola you are picking the simplest knowledge patterns maliciously :) you can upload patterns of any complexity if you like, but you cannot oblige everyone to use all the most complex ones in all projects [10:51] AldoGangemi: @Nicola as I said explicitly before: there are several types of competency questions: I mentioned also reasoning- and stress-oriented questions, and provided testing methods to use them, e.g. as use cases [10:51] MikeBennett: @Aldo surely the main point from a quality point of view is what can be specified and subsequently verified. [10:52] AldoGangemi: @Nicola the planning-scheduling one is just a complex example of modeling issues, with mixed types of questions [10:52] SimonSpero: @MikeBennett: Verified necessary prior, but what about validation [10:52] SimonSpero: (that is, that the verified implemented system meets users actual needs) [10:53] ValentinaPresutti: @Nicola @MikeBennet: I think that too much information is missing from Nicola's slide content about ontology patterns. Also, consider that patterns on the portal are not certified, there must be a motivation :) [10:53] MikeBennett: @Simon good point - let's cover this one in the general discussion, I think it's important. [10:53] AmandaVizedom: @Aldo: There is a lot of space between max complexity / interop and typical ODPs, and @Nicola's position seems to be well short of the extreme at which you are striking. Can we stick to the presented content and not engage in too much filling in beyond this context? [10:55] MikeBennett: I unmuted locally [10:55] AldoGangemi: @Amanda I'm just objecting to some oversimplification of ODPs. Secondly, I'm pointing to the fact that interoperability can mean different things, with degrees in between, yes [10:56] ValentinaPresutti: @SimonSpero @Nicola @MikeBennet: XD is oriented to verification through test-driven design. Tests are driven by requirements, which include reasoning and constraint criteria [10:57] ValentinaPresutti: @Nicola: if patterns need to be all connected you end up in having a huge ontology -> they miss their "being component" thing [10:58] MikeBennett: === AmandaVizedom presenting... [10:59] SimonSpero: @Valentina: but see BDD: http://en.wikipedia.org/wiki/Behavior_Driven_Development [11:00] JenniferWilliams: @ Todd - you asked "to what extent did the need for semantic consistency across components and interfaces impact the design?". In the beginning of the system, not as much as it should have done! This system started development about 8 years ago, and at that time there was not much 'best practice' guidance out there on how to design an ontology-driven system (although probably thinking about the issue in terms of Model-driven Architecture would've helped). Since the modeling team has gotten more involved in providing the teams the implementation artifacts they require, the teams have increasingly designed their code to permit them to "hot swap" the semantics-containing parts of their code - so, less and less of the business logic is hard-coded. [11:00] JackRing: If ontology represents Purpose as well as System then doesn't the quality criteria reduce to 'self-consistency?' [11:00] SimonSpero: Valentina: but sprints, short POD cycles etc are also XP [11:00] SimonSpero: (Agile) [11:00] JenniferWilliams: @ Nicola - you said "The reference ontology which appears in the case being discussed by Jennifer is indeed *application independent* (sort of)" - It's intended to be, although nothing's perfect :-). To the extent possible, we try to develop the Reference Ontology based on our best understanding of the real-world behaviors of the entities involved, and we try to partition "implementation compromises" (model features added to deal with incomplete or dirty data, or model parts we don't have time to fully investigate and 'get right') in a different (logical) part of the ontology (although still expressed in ECLIF, we know this part of the model is in "semantics purgatory" until we have time to clean it up). [11:01] JenniferWilliams: @ Nicola - you asked "@Jennifer: what is the main thing you ask to your reference ontology, in terms of quality?". Coverage - it has to cover the entities needed for each Scenario Model. Easy to verify. Internal logical consistency - the HIGHFLEET XKS helps us check this (e.g. if (p x y) -> (q a b), the type of x has to be consistent with a, and y with b.) Robustness to change - that's what formal ontology is best for - if the modeler tries to consider not just what data you have in hand about an entity, but what that entity really does in the world (definitely using OntoClean-type analysis to help with this) you hope to arrive at formalizations that will accommodate the next system's perspective on the entity. [11:02] JenniferWilliams: @ Todd - you asked "Jennifer, could say a bit more about the 'config' files? What do they configure?". Here's a non-CS-person's perspective; they are intended to convey to the system components the kind of information about the data elements that those components need to manipulate. Say that the Middleware component is responsible for query elaboration - so if users ask about agents that own vehicles, the middleware makes sure the knowledge system asks all the appropriate questions about agent subclasses (maybe organizations and people) and vehicle subclasses - then Middleware needs to know which kinds of entities are permitted to be asked about (via the UI), and how to find their subclasses (the subsumption hierarchy). The middleware config file doesn't know anything about UI display-names-to-users or display icons. Our UI team, in turn, doesn't have anything at all in their code about wha! t are the permitted ontology entities (classes or relations) - they get all that from the "drop-in" config file the Modeling Team provides them. [11:04] ToddSchneider: Jennifer, thank you. [11:04] MikeBennett: Formal soundness checks are actually very useful if one is simply doing an ontology to capture complex business knowledge, since it will catch errors that are easy to make when trying to capture the domain knowledge across a large and complex area of the business. One should expect to find such errors and expect to do soundness checks as a QA step, in my expereience. [11:07] AldoGangemi: @Amanda on physics and chemistry: +1 [11:07] SimonSpero: @MikeBennett, @AmandaVizedom: That's the whole confusion between Lexicon, Controlled Vocabulary, and Ontology [11:08] MikeBennett: @Simon not clear what the "that" is you are referring to here... [11:09] AldoGangemi: @Simon @Mike eheh referential opacity everywhere [11:10] ValentinaPresutti: @SimonSpero: thanks for BDD ref., interesting, but it's about software, I'm referring to ontologies which are less concerned with behavior (at least in that fashion). [11:11] SimonSpero: @Mike: in controlled vocabularies, subsumption across parthood and superclasses *is* transitive (different domain of interpretation). It's when this is missed, and people try to stick a controlled vocabulary into an ontology without understanding the difference [11:11] MikeBennett: @Simon thanks. [11:11] SimonSpero: Unintentionally intensional intentionality [11:12] JackRing: Amanda, how does your Accuracy attribute compare to False Positives and False Negatives? [11:12] PeterYim: @Amanda - on your slide#4, is the last bullet supposed to be under the second last one (or vice versa)? [11:13] LarryLefkowitz: @Amanda: I agree re needing to know when a model is applicable. Taking this further, I'd argue that *all* knowledge (modeled or otherwise) is contextual. (Yes, some contexts are far-reaching: some abstract knowledge, such as mathematics, applies broadly.) Thus, it becomes necessary to specify the domain of applicability for any knowledge. One granularity for doing this is at the level of an ontology. At the other end of the spectrum is to state that each assertion/axiom on that model should be contextualized. At the former end of the spectrum, it is feasible for users to decide when to apply (or not apply) a given model. At the latter, finer-grain end of the spectrum, the context information becomes fodder for the inference process, i.e., inference can determine what information is appropriate to include, depending on the question in which a query was posed. [11:13] JenniferWilliams: @Bobbin: You asked "In what form did you deliver the 'ontology' for use; and was this an interative process, or parallel, or?" - The strongest (ontologically) part of the ontology (the Reference Ontology), and our model of the system component interface specs, are only internally delivered to the Modeling Team, so that's nice, because we don't have the burden of making them user-facing (that is, they can be as complex as is helpful for the modeling team itself to use). The Implementation Artifacts are delivered to our users as the "config" files I tried to describe @Todd above - (and I'm not a computer scientist, so apologies if I'm mangling the proper usage of "config" here). So, e.g., if one part of the Knowledge System engine is optimized to answer specific kinds of question, it only gets a config (an XML file) covering those entities relevant to the calculations _it_ doe! s, and does not contain any information about how the users like to see those entities represented. This also had the nice effect of taking the Modeling Team out of the loop for some aspects of the system that need to be tied to the semantics but which are not themselves modeling tasks - I'm thinking of the way we set up the UI team's config so that they could "write in" the (ever-changing) user-preferred display terminology without asking the Modeling Team to do it. We (the modeling team) don't care what the users call the system entities; as long as all names tie back (in some config) to a semantically-sanctioned entity so the Test Team can verify system operation. [11:15] BobbinTeegarden: @Jennifer Thanks! [11:15] ValentinaPresutti: @SimonSpero: still about BDD - actually without knowing about BDD we included some of the principles in XD such as stakeholder involvement :) [11:16] CoryCasanave: Strong agreement: There is often confusion about the subject of a model/ontology in reference to a model of the domain or information about the domain or a system in the domain. Is there a "meta ontology" to express this? [11:16] AldoGangemi: @Amanda indeed NL plays a big role not only in ontology design from text, but also in matching requirements to solutions, and in evaluating domain coverage [11:17] JackRing: Peter, Why am I stuck on Slide 3 even after several Refresh? [11:17] MikeBennett: @Jack the slides are advancing OK here - check you are not looking at a local copy of the slides? [11:17] PeterYim: @JackRing - can you refresh browser (to reload the viewer) and try again? [11:17] SimonSpero: @Valentina: That shows up in the Agile world in the "Scrum" approach. I hate the terminology, but the emphasis on short cycles with return to stakeholder/SME's, the daily stand-ups, the pigs & chickens, etc [11:21] AldoGangemi: @Amanda: "let a crowd of researchers know wgat you need and why: you may soon find that you have it": great, exactly what happens in the most efficient commuity, such as the linked data one. Headline: stress collaboration rather than conflict [11:22] SimonSpero: Scrum eg: http://www.scrum.org/ [11:22] SimonSpero: If you know anything about rugby, forget it [11:22] AldoGangemi: apologies, too many typos, it's late ... [11:24] ValentinaPresutti: @SimonSpero: sure, agile software development has been our inspiration but there are specific issues on ontology design that ask for specific solutions. I have to go through BDD a bit more specifically for understanding possible analogies to ontology design [11:25] JackRing: Caution, agile development rarely produces agile products. [11:25] SimonSpero: I asked how short development cycles should be [11:25] SimonSpero: Jennifer says they are using 90 day PODs [11:26] SimonSpero: Jennifer says that too short makes things too specific [11:26] NicolaGuarino: @Jack: +1 [11:31] SimonSpero: [I don't get to ask two questions live, but I had one less specific one: When assessing needs and correspondence- how do you determine when you need an ontology, when do you need a thesaurus or faceted classification system (controlled vocabularies whose is documents) [11:31] ValentinaPresutti: need to go, thanks everybody :) [11:31] SimonSpero: (whose domain of interpretation is documents - chat strips out greek letters) [11:32] MikeBennett: We are a little over time - do we have time to take Jennifer's comment after Nicola? [11:32] MikeBennett: Thanks @Valentina [11:32] AmandaVizedom: @Mike: Let's try. [11:33] AldoGangemi: @Simon I think there can well be cases when you just need skos-like structures, e.g. for text classification, basic semantic search, lightweight text annotation [11:33] BobbinTeegarden: @Jack, Nicola Depends on how visionary, how good a conceptual integrator your facilitator is to some degree, as an enabler for shorter cycles with abstract integration, while maintaining usefulness and quality. No? [11:33] PeterYim: @Mike and @Amanda - since we started a little late, feel free give yourself another 7 minutes (from now) [11:33] JenniferWilliams: I can type it instead - just going to say to compare two models, you need to look at any extant model doc'n, but also any existing data (which shows the _actual_ model sanctioned by the data system), and ideally use the UI too (which shows the way the data is manipulated by the system). All informative. [11:34] MikeBennett: Thanks @Peter [11:35] SteveRay: Have to run. Interesting session. Thanks. [11:35] SimonSpero: @Aldo Yes - though the final version of SKOS got confused (huge time gap between draft and final) [11:35] AldoGangemi: Indeed [11:35] AliHashemi: @Nicola - An observation about the issue regarding design patterns and interoperability. I'm not sure that a design pattern necessarily needs to be underspecified or over generalized. It seems to me that they may span various levels of complexity. Even then, a design pattern with very limited ontological commitments (i.e. the part pattern example which simply stated it was a transitive relation) - can still ensure that sentences containing the use of said relation satisfy the constraints as specified in the pattern. Really, it seems the issue is with ontologies that contain significant semantics as opposed to design patterns per se. Which perhaps is related to the quality of such patterns. [11:35] LeoObrst: @Simon: controlled vocabularies and thesauri are lexical resources, as opposed to conceptual resources. [11:37] SimonSpero: @Leo: CVs fall in the middle ground (part lexical, part conceptual) - the standards are all defined as relating lexemes to intentional concepts [11:37] LeoObrst: @Simon: we (I) use controlled vocabularies (and thesauri) to go from the natural language that users use to the ontologies which represent the concepts (referents) needed, i.e., the ontologies represent what the lexical notions "mean". [11:37] AmandaVizedom: @Simon: I would try to spell out the functional requirements needed, avoiding using the ontology/thesaurus/fcs boundaries specifically, and to enumerate what the broken out ontology quality factors might are so that the relationship between the oqfs and requirements can be better examined and understood. [11:37] MikeBennett: Thanks @Amanda for pulling it all together! [11:37] AldoGangemi: @Ali exactly: let's be tolerant on having patterns of different complexity for different uses [11:37] PeterYim: -- session ended: 11:37am PST -- [11:37] LeoObrst: @Simon: there's more to say, of course. [11:38] AldoGangemi: Great session, thanks Peter [11:38] AldoGangemi: Mike, Amanda, all [11:38] LeoObrst: Good session, all! [11:38] PeterYim: Thanks, everyone! [11:38] SimonSpero: @Leo: Definitely! ----------------