Hi Mike, (01)
Thanks for the distinctions and especially for expressing - "Business Value" (02)
Paul Strassmann attempted to quantify the "Business Value of
Computers" and of those who manage investments in his 1990 book. I
wonder what use these concepts of Value might have in the current
discussions? (03)
Regards, (04)
Bob (05)
Bob Smith
Professor Emeritus - CSU
Tall Tree Labs (06)
On Sun, Feb 12, 2012 at 8:07 AM, Mike Bennett <mbennett@xxxxxxxxxxxxxxx> wrote:
> The term "quality" may mean different things to different people.
> I want to draw out two of those possible senses and kick off some
> discussion about the possible connections between them.
>
> Here I am not trying to introduce anything new, I'm trying to
> describe what's in the relevant literature. So these definitions
> are not really discussion points, but people are welcome to
> clarify and correct any vagueness with reference to the literature.
>
> The two senses I would characterize as:
>
> 1. Quality in the dictionary sense of the word: how good
> something is;
> 2. Quality in the sense used in the industrial term "Quality
> Assurance".
>
> In the second sense, Quality Assurance (QA) simply describes an
> approach whereby one can ensure (and demonstrate that one has
> ensured) what the qualities of some deliverable are. QA is not
> about making better things, it is about better making things.
> It's about metrics and measures.
>
> The basic definitions and parameters of QA are defined in QA
> standards such as ISO 9000. They are also described in national
> standards such as BS5750 or the German SUV system.
>
> Quality Assurance is about being able to "demonstrate control".
> That is, a firm has formal processes in place by which they
> manage their deliverables. These processes are designed and
> optimized to ensure consistency in the process of creating
> whatever their deliverables are.
>
> We could refer to (1) and (2) as "Qualitative Quality" and
> "Quantitative Quality" (you see why the choice of the word
> Quality wasn't a good one!)
>
> To illustrate the difference: Macdonald's Golden Arches has
> arguably the best QA system of any restaurant chain. However, few
> would argue that they make the best burgers. Qualitative quality
> doesn't necessarily follow from quantitive quality. What does
> follow is the business value in consistency. Anyone who finds
> themselves in a strange city with hungry children in tow will not
> take them to an unknown burger chain where they food may or may
> not be excellent; they will take them to the place where they
> know exactly what they will get. This is the business value of a
> good QA system.
>
> An accurate if tongue in cheek characterization of QA is given by
> Scott Adams in the Dilbert cartoons: "Say you're going to produce
> crap; produce crap; prove you produced crap" (please excuse the
> language). In fact, if you substitute anything at all (any
> quality at all) where Adams has the word 'crap', you have a
> reasonable summary of how QA works. You formally specify the
> things you are going to produce; you produce those things
> according to a well defined (and auditable) set of formal
> processes, and you end up with an auditable set of records
> demonstrating exactly how those things were produced. This is why
> the phrase "Demonstrate control" is the heart of QA.
>
> Now you could of course specify that you are going to produce
> excellent things. Rolls Royce for example has a reputation for
> doing this. Once the firm knows where its market is, and what
> sort of things it wants to and is able to produce for that
> market, then it simply formalizes the qualities that it wants to
> produce in its products or services.
>
> How might we apply this to ontologies?
>
> To the extent that different qualities of an ontology can be
> formally specified, these can be input to a formal, industrial QA
> process.
>
> To the extent that one can formalize Quality (1), that is "what
> makes a good ontology?", one can apply Quality (2) QA to ensure
> that ontologies are produced which comply with those requirements.
>
> There are other things one might want to ensure about the
> deliverable ontologies. For example if one is extending an
> ontology which is built according to certain microtheories, one
> would want to ensure that those microtheories are consistently
> applied in the new material. Sometimes this can be detected by
> simple measures such as consistency checking; sometimes perhaps
> it may not.
>
> Validation and Verification:
>
> These terms also have specific meanings when applied to formal,
> industrial QA systems, which are narrower than their dictionary
> sense. In software deliverables these translate to:
>
> Verification: Testing the deliverable against its formal
> specification. Test cases verify, against each functional
> requirement in the specification, that that requirement is met.
> Validation: Ensuring that the system as specified and delivered,
> actually meets the customer's expectations (typically User
> Acceptance Test). Here we find out whether the specify-build-test
> set of activities actually resulted in what the customer wanted.
>
> How these are implemented depends very much on the technology and
> the architecture of course. There may be interesting challenges
> in applying both of these for ontologies.
>
> Design review / peer review
>
> The word "Peer review" has a slightly different meaning in
> industry, to that which it has in academia. A QA process does not
> just consist of tests; frequently one designs into the process
> some kind of review activity (typically a meeting), often called
> a design review, also referred to as a form of peer review. This
> is where some deliverable item in the QA process (usually a
> design specification) is presented to a group of people who would
> be in a position to critique it and verify that it is fit for
> purpose. Part of the design of the QA process is to consider the
> knowledge requirements for various forms of design review (e.g.
> system architects, the QA department, coders in the appropriate
> language), and ensure that at the relevant point in the process,
> the right knowledge is gathered around the table for that
> deliverable to be signed off. Another form of this is the code
> walk through.
>
> Again, there would be similar but different applications of this
> thinking in the various stages in development of an ontology.
> Perhaps you might review the taxonomy (or apply some of the
> structural tests described in the literature), before developing
> the whole ontology. Or not. A QA process has to be designed, it
> doesn't just happen.
>
> People often think that the imposition of a formal QA regime on a
> technical development implies that some "waterfall" development
> model must be applied. This is not the case. If you look at other
> formal processes such as the Rational Unified Process (RUP), you
> will see the same linkages between deliverables, applied such
> that the deliverables can be changed and delivered to a faster
> time scale. Similarly the "Agile" approach, when examined in
> detail, shows that those linkages are still in place.
>
> For ontologies, if these are to be updated freuquently and in
> real time, even more imaginative ways of applying the basic
> parameters of QA might be required. For example, one might choose
> to use a "gardening" type of approach, whereby the QA is applied
> after the event by making or proposing updates or corrections.
> Here the QA process followed by Wikipedia is a good example of
> how this may be done.
>
> Anyway I realise that not everyone has been thinking about
> Quality (2) / quantitive quality, and this is just one part of
> the bigger picture. But I hope that, for those who have spent
> less time in an industrial environment, this is a useful guide to
> what those of us from such an environment usually mean when we
> talk about Quality Assurance.
>
> The key point is that the more things about quality in the
> natural language, qualitative sense, can be formalized, the more
> these can be made use of in formal, industrial QA.
>
> Hope this is helpful,
>
> Mike
>
>
>
>
> --
> Mike Bennett
> Director
> Hypercube Ltd.
> 89 Worship Street
> London EC2A 2BF
> Tel: +44 (0) 20 7917 9522
> Mob: +44 (0) 7721 420 730
> www.hypercube.co.uk
> Registered in England and Wales No. 2461068
>
>
> _________________________________________________________________
> Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
> Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
> Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
> Community Files: http://ontolog.cim3.net/file/work/OntologySummit2012/
> Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2012
> Community Portal: http://ontolog.cim3.net/wiki/ (07)
_________________________________________________________________
Msg Archives: http://ontolog.cim3.net/forum/ontology-summit/
Subscribe/Config: http://ontolog.cim3.net/mailman/listinfo/ontology-summit/
Unsubscribe: mailto:ontology-summit-leave@xxxxxxxxxxxxxxxx
Community Files: http://ontolog.cim3.net/file/work/OntologySummit2012/
Community Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OntologySummit2012
Community Portal: http://ontolog.cim3.net/wiki/ (08)
|