OntologySummit2011: Value Metrics and Value Models (Track-3) community input    (2LKU)

New:    (2OFS)

26 February 2011: Questionnaire for Use-Cases: Problem-Solution-Metrics Matrix    (2OFT)

Collecting Use-Cases - Connecting to Intended Audiences - Measuring Value    (2OFY)

http://ontolog.cim3.net/file/work/OntologySummit2011/ValueMetrics/UseCaseMatrixExample--RexBrooks_20110226a.jpg    (2OKJ)

Use-Case shown above corresponds to the first entry in the Questionnaire for Use-Cases below.    (2OH4)

Track 1 Ontology Application Framework -- Application Categories    (2ON8)

It is requested that your Use-Case Name be accompanied by an Application Category from the four detailed in the Ontology Application Framework from Track 1:    (2OND)

Please Note that Track 3 Value Metrics-focused Use-Cases are condensed and meant to be gathered together in tabular form to highlight Metrics. While it will include material from Track 2 Case Studies, it is not intended to be a Full Use Case. For that we provide links to those full Use-Cases    (2OMW)

Connecting to the Full Track 2 Case Studies    (2OMX)

Case Study Summaries Each Case Study participant was asked to provide a grid on one slide, outlining the business problem, the solution, key features (or screen shot) and business benefits. The aim of this was to be able to identify what sort of "Ontology" this was in terms of the application framework once this was completed, and what metrics (if any) were avilable to determine the business benefits. (2OR6)    (2OZO)

Each Case Study (2OR7)    (2OZP)

    * Challenge    (2OR8)
          o What business problem the Ontology set out to address    (2OR9)
    * Solution    (2ORA)
          o What we mean by ontology in this case e.g. application, conceptual model    (2ORB)
    * Screen shot or key features    (2ORC)
          o Give a flavor of the ontology    (2ORD)
    * Benefits    (2ORE)
          o What metrics if any were used to demonstrate the benefits of this ontology.    (2ORF)    (2OZQ)

Integration of Multiple Systems from Multiple Companies YefimZhuk, Sallie Mae (2OT1)    (2OZR)

Challenge (2ORH)    (2OZS)

    * Multiple systems and sources of knowledge in different parts of the enterprise, owned by different communities of practice.    (2ORI)
    * Gaining time and commitment from subject matter experts to ensure completeness of the model.    (2ORJ)
    * Different groups see different shades of meaning and application for similar terms, in different contexts.    (2ORK)
    * Needs a unifying approach supporting local views    (2ORL)    (2OZT)

Key Ontology Features (2ORM)    (2OZU)

Solution (2ORN)    (2OZV)

    * Facilitation of knowledge gathering using ontology engineering methods.    (2ORO)
    * Formal ontology notation for single ontology, while presenting views and facets of this to subject matter experts.    (2ORP)
    * Curation of the ontology    (2ORQ)    (2OZW)

Benefits (2ORR)    (2OZX)

    * Best use of subject matter experts¡¯ time and resources    (2ORS)
    * Curatorship of Enterprise Semantic Architect ensures quality, consistency and completeness of the ontology    (2ORT)
    * Collaboration in industry standardization efforts (e.g. EDM Council), via common semantics    (2ORU)
    * Ensures that the knowledge captured at Sallie Mae is taken forward to industry-wide standardization efforts which we can then use    (2ORV)    (2OZY)

Standardization of Terms and Definitions for Financial Services].    (2ON3)

Specifying Metrics in the Use-Case    (2OMI)

Please specify both Quantitative and Qualitative Metrics. Such Metrics should include, but are not restricted to metrics cited in Track 1 Ontology Application Framework:    (2OMK)

If using existing Use-Cases please give Metrics cited by the authors of the Use-Case if available.    (2OMV)

If possible be specific, such as ROI, and explain or define the term for the Metric if not well known.    (2OMJ)

16 February 2011: Initial Community Input Page    (2OH2)

Questionnaire for Use-Cases: Problem-Solution-Metrics Matrix    (2OG1)

Note: Add your Use-Case per the following instructions:    (2OG4)

Please login, click on "Edit text of this page" at bottom of this page, then, in the editing box, copy the entire section that follows down to the line that ends immediately above the heading "Add new Use-Case Below", then paste it into the editing box just below that same heading and fill-in the information for the Use-Case you are adding. Then click on "Preview" to check your Use-Case and once satisfied that it is complete, please click "Save" with the Summary "Added (name) Use-Case."    (2OG3)

Use-Case Name and OAF Category if appropriate:    (2OGC)

Replace this text with Use-Case Name.    (2OG6)

Use-Case Problem(s) & Description:    (2OGD)

Replace this text with Use-Case Problem(s) & Description. Please be specific, concise & keep in mind that this Track is aimed at Metrics.    (2OG7)

Use-Case Solution Used or Proposed:    (2OGE)

Replace this text with Suggested or Expected Use-Case Solution. Please ask yourself what kind of solution is needed from the viewpoint of the Business Case?    (2OG8)

Is an Ontology-Based Solution really called for in this Use-Case? Why or Why not?    (2OG9)

Metrics needed to measure the success of Used or Proposed Solution to the Use-Case Problem(s):    (2OGF)

Replace this text with specific details of metrics. What is measured? How is it measured? What does the Metric show?    (2OGA)

What is the Intended Audience & How do we reach them:    (2OGG)

Replace this text with the name and description of Intended Audience. Please be specific, with particular attention to the kind of language or vernacular understood by the intended audience and include opinion about appropriate means to capture attention of the audience.    (2OGB)

What Else needs to be learned from our Use-Cases:    (2OGJ)

Replace this text with suggestions, thoughts & brief discussions for what else needs to be learned from our use-cases in addition to metrics to help us make the case for ontology.    (2OGK)

Add new Use-Case Below:    (2OGL)

(Note: Each Use-Case will be a row in a typical spreadsheet type matrix with each question comprising a column, as shown in Figure 1 above. Additional 2D matrices may be added using different, but related, questions for columns as shown in Figure 2 above.)    (2OGM)

Use-Case Name and OAF Category if appropriate:    (2OGC)

2006CRM-TopTierIT-ProviderFor-AutomotiveAfterMarketRetailer    (2OG6)

Use-Case Problem(s) & Description:    (2OGD)

Eroding Customer Loyalty & Aging Technology requires Retailer to Improve Customer Shopping Experience.    (2OGN)

Use-Case Solution Used or Proposed:    (2OGE)

"Open Standards (EDI-SQL-based RDBMS) and up-to-date (J2EE-based Platform) "Flexible, Web-based" IT system used for company-wide "Integration Framework."    (2OGO)

NO Ontology-Based Solution called for in this Use-Case? Why or Why not is a guess since the example did not mention this at all, though the use of an EDI-SQL RDBMS provides a de-facto semantically consistent Datamodel.    (2OG9)

Metrics needed to measure the success of Used or Proposed Solution to the Use-Case Problem(s):    (2OGF)

What is the Intended Audience & How do we reach them:    (2OGG)

What Else needs to be learned from our Use-Cases:    (2OGJ)

The fact that some hidden semantic consistency in both the open standard terminologies used and in the terminologies of the Platform used can be cited, but the problem with this is that it is not very specific to the metrics used to measure the success of this solution.    (2OH1)

Add new Use-Case Below:    (2OGL)

(Note: Each Use-Case will be a row in a typical spreadsheet type matrix with each question comprising a column, as shown in Figure 1 above. Additional 2D matrices may be added using different, but related, questions for columns as shown in Figure 2 above.)    (2OGM)

Use-Case Name and OAF Category if appropriate:    (2OGC)

Digital Music Archive (DMA) for the Norwegian National Broadcaster (NRK)    (2PBA)

Knowledge Management    (2PBB)

Use-Case Problem(s) & Description:    (2OGD)

Public broadcasters have large archives ranging back 60+ years including sound assets on bakelite, vinyl and wax. Some older assets show remarkable longevity, but modern storage formats like digital video tape, certain CDs, tapes, etc are not as robust. At NRK many tapes recorded in the late 80s and early 90s could be recovered within 5 years without immediate action tor preserve these assets for the future.    (2PBC)

Use-Case Solution Used or Proposed:    (2OGE)

Model the Repository using semantic web technology (XML-based business rules), including transcription of metadata from well-structured, high-quality paper-based non-relational analog system to digital, semantically aligned, relational database system while completely revamping entire radio and television broadcasting production process and remastering library of recordings.    (2PBD)

An ontology-based solution was necessary, albeit couched in Semantic Web terminology. However, with an estimated 150+ million triplestore anticipated, a semantically-aligned RDBM was implemented to scale up to a Semantic Web based publication layer for the user interface..    (2PBO)

Metrics needed to measure the success of Used or Proposed Solution to the Use-Case Problem(s):    (2OGF)

It continued to be tested to evaluate scalability of available systems as of 2007. Specific tests and results were not given.    (2PBF)

Success is measured against expected benefits:    (2PBG)

What is the Intended Audience & How do we reach them:    (2OGG)

Intended audience appears to be consumers of Norwegian Public Broadcasting.    (2PBM)

What Else needs to be learned from our Use-Cases:    (2OGJ)

Of particular interest is the fact that this archive was specifically designed to be compatible with future accessing technologies, specifically SPARQL. The fact that this example failed to note the metrics used indicates that as of 2007, metrics were not deemed sufficiently important to mention.    (2PBN)

Add new Use-Case Below:    (2OGL)

(Note: Each Use-Case will be a row in a typical spreadsheet type matrix with each question comprising a column, as shown in Figure 1 above. Additional 2D matrices may be added using different, but related, questions for columns as shown in Figure 2 above.)    (2OGM)

Use-Case Name and OAF Category if appropriate:    (2OGC)

Suppliers' Benefit from IT Use in Supply Chain Relationships    (2Q2X)

Decision Support    (2Q2Y)

Use-Case Problem(s) & Description:    (2OGD)

Supply chain management systems (SCMS) championed by network leaders in their supplier networks are now ubiquitous (2004). Demonstrating the benefit to the supplier needs to be established.    (2Q2Z)

Data from 131 suppliers using an SCMS implemented by one large retailer support hypotheses that relationship-specific intangible investments play a mediating role linking SCMS use to benefits.    (2Q30)

The results support the vendors-to-partners thesis that IT deployments in supply chains lead to closer buyer-supplier relationships.    (2Q31)

Use-Case Solution Used or Proposed:    (2OGE)

This Use-Case comes from 2004 so that needs to be understood.The solution examines and depends on the "intangible" value of exploitation and exploration by suppliers in the Integrated IT supply-chain management relationship.    (2RPN)

An Ontology-Based Solution was called for in this Use-Case, but this is not an endorsement of OWL at large. The reason why is because the value of semantic consistency which is needed for any solution to this problem topic is only now becoming more or less widely accepted. In the supply chain scenario, it is essential that buyer and seller use the same well-defined and understood terminology. Understanding the "intangible" values in exploitation and exploration is the solution proposed through the use of an integrated IT from a top tier vendor    (2Q34)

Understanding the "intangible" values in exploitation and exploration is the solution proposed through the use of an integrated IT from a top tier vendor.    (2Q35)

Metrics needed to measure the success of Used or Proposed Solution to the Use-Case Problem(s):    (2OGF)

This Use-Case was chosen in part because it is available online in order to allow us to look at how Relationship-Specific Intangible Assets are measured in fairly complete detail.    (2Q36)

What is the Intended Audience & How do we reach them:    (2OGG)

The Intended Audience for this Use-Case Solution is top management decision support who must select and justify their selection for a top tier IT SCMS solution. However, for us, this Use-Case can be used to show how qualitative values can be measured as quantitative results from survey-based research.    (2Q37)

What Else needs to be learned from this Use-Case:    (2OGJ)

This Use-Case provides subtle validation for the use of ontology, though only by deduction from the use of the implied semantics of open standards, as in the first Use-Case, in defining a system to measure qualitative, strategic value for suppliers who use the top tier IT solution. This use of indirection is something we should consider. However, this should be understood in context, to quote from the Use-Case "Within the context of an exchange, specialized, relationship specific assets create more value than nonspecialized, generic assets."    (2Q38)

For our case, should we extend this by substituting the word "ontology" for "assets"/    (2Q39)

Add new Use-Case Below:    (2OGL)

(Note: Each Use-Case will be a row in a typical spreadsheet type matrix with each question comprising a column, as shown in Figure 1 above. Additional 2D matrices may be added using different, but related, questions for columns as shown in Figure 2 above.)    (2OGM)

Use-Case Name and OAF Category if appropriate:    (2OGC)

Resource List Management (RLM)    (2QLE)

Knowledge Management    (2QLF)

Use-Case Problem(s) & Description:    (2OGD)

Resource Lists are collections of text books, journal articles, and other content defined by instructors for students. Once, these were provided as paper handout lists requiring students to visit various suppliers for the resources. Now, these materials are available over the Web, and in an e-delivery model.This move to electronic resources in itself has improved the access landscape somewhat. However, as an institution shifts subscription agents, the landscape for access rights becomes complex and changes frequently. Link resolver solutions go some way to solving these issues, but are potentially confusing.In addition, the interoperability of data between publishing platforms, the University library catalogue, the University Virtual Learning Environment (VLE) and RLM tools themselves has been poor.    (2RPO)

Use-Case Solution Used or Proposed:    (2OGE)

As well as re-using existing ontologies, Provider developed and published two new ontologiesn and a web-based delivery system as part of the project. The Resource List ontology [1] underpinned the semantics of the relationships between resources and intended uses. The Academic Institution Internal Structure ontology (AIISO) described the courses, modules, departments and schools that make up an institution, which was required to enable instructors to link lists to the relevant module or course, to enable students to find lists.    (2QLJ)

Is an Ontology-Based Solution really called for in this Use-Case? Why or Why not?    (2OG9)

Ontology technology could be used in the following areas:    (2QLK)

For the instructor, other enhancements aim to make harvesting resources and the creation of lists much easier. (One assumes from the use of "harvesting" and other references edited out of this format that a custom inferencing engine is included.)    (2QLP)

Using a bookmarklet in a similar pattern to sites such as del.icio.us, instructors can harvest resources from a multitude of Web sites, including the library catalog. The difference between this system and most bookmarking services was claimed to be that the system attempts to identify the resource the page is describing, rather than just recording the location of the page itself. This system evaluates microformat-style markup such as COinS to pull metadata from the page. Once obtained, the metadata is stored in the instructor’s library as RDF using the Bibliographic Ontology, thus increasing the interoperability of the harvested data with other systems and workflows.    (2QLQ)

Metrics needed to measure the success of Used or Proposed Solution to the Use-Case Problem(s):    (2OGF)

This example is characterized most by the lack of metrics, except what one can infer from the fact that the system was launched at the University of Plymouth, one of the existing focus group partners, in September 2008, initially with just 1000 students. Throughout the autumn semester, Plymouth increased adoption of the system with the aim of giving access to all 22000 students in early 2009.    (2QLR)

However, we have no data on the cost, nor metrics on the effectiveness, though there is a bulleted list of "Benefits."    (2QLS)

What is the Intended Audience & How do we reach them:    (2OGG)

University Administrators, Professors and Adjunct Instructors were the intended targets but no information was included in the Case Study to show how these audiences were reached, beyond citing that the case study surrounding the system itself was conducted "In partnership with 15 UK and Eire Universities (the focus group)..."    (2QLT)

What Else needs to be learned from our Use-Cases:    (2OGJ)

The fact is that this could have been an excellent Use-Case if metrics had been established beforehand for tracking a whole host of datapoints from the cost savings to the University from its previous system to ROI over a period of years as well as customer satisfaction surveys of instructional staff and students.    (2QLU)

Use-Case Name and OAF Category if appropriate:    (2OGC)

Supply Chain Quality of Service (QoS)    (2QSB)

Knowledge Management was the domain cited in 2004, but in hindsight it could apply more to Decision Support.    (2QSC)

Use-Case Problem(s) & Description:    (2OGD)

Retailer must optimize its supply chain on the basis of timely supplier quotes via web-service based on its quality of service criteria. Selection criteria for supply chain decisions are needed for the dynamic composition of late-binding Semantic Web Services. A key criterion is quality of service (QoS).    (2RPP)

Traditionally, supply chains are static with close collaborations between suppliers and retailers. Recently, exchanges and auctions have added dynamism to these chains. Web services allow the creation of dynamic supply chains. The high cost of integrating retailer's forecasting software with supply chain management is a hurdle, but web services combined with semantic technologies offers a less costly solution. The web service to gather quotes from a number of suppliers with near real-time acquisition of domain constraints, behavioral signatures, and Quality of Service (QoS) parameters. Hence the need for dynamic composition of semantic web services.    (2RPQ)

Use-Case Solution Used or Proposed:    (2OGE)

The proposed solution requires the use of QoS in conjunction with semantics for creating and optimizing dynamic, realtime business intelligence processes. Domain specific ontologies are crucial for agreement between retailers and suppliers about domain specific parameters. This use case uses ontologies and semantic metadata for representation and automated discovery of candidate services. This is necessary to provide service selection criteria to use for decision making from amongst the group of suppliers a retailer may have for a given manufactured part or raw material. A generic QoS ontology (currently a prototype is under development in the METEOR-S project http://lsdis.cs.uga.edu/proj/meteor/ ) is needed. (This has undergone further work from the point it was mentioned in this Use-Case from 2004. The semantic consistency of BPEL4WS is assumed.    (2RPR)

Metrics needed to measure the success of Used or Proposed Solution to the Use-Case Problem(s):    (2OGF)

QoS criteria can either be generic or domain specific. Generic QoS metrics, a standard component of these metrics involves time, cost and reliability. There is a need for a well accepted ontology for the generic QoS criteria for candidate suppliers..Specific criteria relevant to service selection are part or material delivery time and part- or material-specific details like reliability of the part or purity of material. These must fulfill the requirements of the Manufacturer.    (2RPS)

Specifically, Web Service QoS metrics of delay, jitter, packet loss rate, and availability were cited. Measurement was further defined as Constraint Analysis. This leads to a deduction that mechanical performance of a web service that otherwise met the criterion of supplying up to date supplier quote was the main metric to be evaluated.    (2RPT)

What is the Intended Audience & How do we reach them:    (2OGG)

The stated audience in 2004 was "Retailer wants to choose the best suppliers based on its business logic. The suppliers want to maximize their profit as well as increase their business with the retailer." So generic retailers and suppliers were the stakeholders considered in 2004.    (2QSN)

What Else needs to be learned from our Use-Cases:    (2OGJ)

Because this was a hypotehtical Use-Case from 2004. it is dated and, apparently, no follow-up has been done, though the METEOR-S project has continued within its academic program. However, this Use-Case is of considerable value to us because, unlike several others, metrics were taken into consideration at the inception of the project.    (2RPU)

Add new Use-Case Below:    (2OGL)

(Note: Each Use-Case will be a row in a typical spreadsheet type matrix with each question comprising a column, as shown in Figure 1 above. Additional 2D matrices may be added using different, but related, questions for columns as shown in Figure 2 above.)    (2OGM)