ontolog-forum
[Top] [All Lists]

[ontolog-forum] Data, Silos, Interoperability, and Agility

To: "'[ontolog-forum] '" <ontolog-forum@xxxxxxxxxxxxxxxx>
From: "Hans Polzer" <hpolzer@xxxxxxxxxxx>
Date: Mon, 16 Sep 2013 23:09:18 -0400
Message-id: <017901ceb353$4cbb8be0$e632a3a0$@verizon.net>
Paul,    (01)

In case it wasn't obvious from my previous responses, my net-centric
assessment practice when I was at Lockheed Martin was focused on the very
points you make. Net centricity is inherently "anti-silo".      (02)

The SCOPE model was developed to help people see the degree of "silo
thinking" they were implicitly exhibiting, and to help them be more
conscious of the scope decisions they were making, and to make them
explicitly, rather than fall into them implicitly. But I did earn the
nickname, the SCOPE Creep, because it is very easy to focus on net-centric
interoperability as an end in itself, and pushing "desilofication" can be
seen as indistinguishable from open-ended scope creep, especially when
viewed from the perspective of the individual system sponsor or project
manager. That's why one of the key net-centric principles we developed was
that of "pragmatism" - the purpose of being net-centric was ultimately to
achieve greater operational effectiveness by leveraging
information/resources accessible via a network connection, including those
one doesn't own or directly control. I think that should resonate with your
statement that " the ultimate measure of system effectiveness is how well it
(they) support(s) the human activities within the enterprise", although I
would not restrict it to "within the enterprise". So much of what
enterprises do these days is done in collaboration with partners, some more
tightly coupled and critical to enterprise objectives than others.    (03)

Having said all that, I was reacting to what I perceived as a common meme
that silos, however ill-defined, are "bad", and silo "busting" was good.
There is even a company out there named "Silobusters", which ironically,
focuses on the CRM silo (in part as an aid to better integrating other
corporate silos with each other). My intent was to get people to move away
from silo-bashing and implying that the problem was the existence of silos,
(not a useful selling point to managers/sponsors of silos, by the way), or
the width/size of silos (as in making everything "enterprise level"), and
more towards the notion of facilitating silo interoperability and
translucence (full transparency scares the heck out of a lot of people, even
those who push it internal to the enterprise - and for good reason).     (04)

More importantly, I have been trying to get people to not stop at the
enterprise boundary (a wider/bigger silo is still a silo) and think about
operations across enterprise boundaries to improve overall enterprise
effectiveness. Talking about making boundaries go away (as in talk of
"seamlessness"), doesn't help here, unless you are talking about
acquisitions, mergers, or integration by fiat/iron fist (i.e., government
"mandate"). Even then, internal boundaries will most likely continue to
exist.     (05)

The elimination of boundaries just doesn't scale as a general solution to
interoperability. Rather we need to focus on how to facilitate crossing
boundaries, whether they be silos, domains, enterprises, or ontologies. The
SCOPE model name is an acronym that plays on this point. And to facilitate
crossing boundaries, we first need to be aware of all the significant scope
boundaries and context assumptions we are crossing (or are willing to cross)
when we approach an operational objective for any given institution, system,
or commercial enterprise. That's the purpose of conducting SCOPE workshops
with domain and enterprise stakeholders who want to interoperate more
effectively with each other and the larger networked environment.    (06)

Hans    (07)

PS: I changed the subject line per Kingsley's suggestion    (08)

-----Original Message-----
From: ontolog-forum-bounces@xxxxxxxxxxxxxxxx
[mailto:ontolog-forum-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Paul Tyson
Sent: Monday, September 16, 2013 10:04 PM
To: [ontolog-forum]
Subject: Re: [ontolog-forum] ONTOLOG community event planning and scheduling
session - Thu 2013.09.12 & Thu 2013.09.19    (09)

Hans,    (010)

I don't disagree with most of what you say, but it is somewhat aside from
the point I was trying to emphasize, which is that the ultimate measure of
system effectiveness is how well it (they) support(s) the human activities
within the enterprise. Of necessity, "silo"
applications are deployed to support specialized functionality for different
parts of the enterprise. That's fine, except:    (011)

1. It often leads to violation of the DRY (don't repeat yourself) principle
for data and business logic.
2. The data available to a user of a "silo" application is arbitrarily
truncated to what the system designers consider necessary to its users.
In a complex, changing enterprise those limitations are at best a nuisance,
at worst an impediment to business process execution.
3. Inbound and outbound links, which might ameliorate the above defects, are
missing from "silo" applications, either because the architecture does not
support them, or to protect vendor lock-in (or both). Good for the software
vendors, bad for the software users.    (012)

Regards,
--Paul    (013)

On Sun, 2013-09-15 at 21:32 -0400, Hans Polzer wrote:
> Paul,
> 
> Remember that enterprises are silos as well, when viewed from the 
> perspective of other enterprises, just a bit wider in scope than silos 
> that are constituent parts of an enterprise. Also, most 
> enterprises/institutions have sub-silos that are significantly 
> segregated from each other for a variety of legal and business 
> reasons, most notably, those enterprises/institutions that operate 
> internationally or in different jurisdictions with significant 
> differences in laws/regulations that impact the operations of the
enterprise/institution. No amount of "silo-busting"
> within the scope/power of the enterprise will change that externality. 
> Some enterprises even deliberately set up competition among their 
> divisions, encouraging "silo" behavior across a significant portion of 
> each division's "operational space", even if other portions of 
> operational space are mandated as shared or common services.
> 
> I'll also note that nothing much gets accomplished without silos. All 
> enterprises, systems, projects are scope limited in operational space, 
> time interval, and most importantly, budget/revenue.  That's true even 
> for individuals and "open-source"  or "crowd-sourced" projects. Put 
> more positively, silos are focused and goal oriented. They also may 
> have information about their internal environment (like personnel 
> information) and about the external environment (like customer info) 
> that can't be shared for regulatory or competitive survival reasons. 
> Even for information that could be shared externally, they may 
> represent it internally in ways that optimize their institutional 
> objectives (which might include the interests of supply chain 
> partners, for example), rather than satisfy the way some arbitrary 
> entity external to the enterprise might want to access or understand that
information.
> 
> Silo behavior is neither good nor bad without taking the larger 
> environment, operational context, and institutional objectives into 
> account. That's why there are corporate mergers, divestitures, 
> re-engineering, and re-organizations, not to mention bankruptcies and 
> start-ups - all forms of silo formation and reformation and 
> dissolution. I'll note that most silo dissolution is not viewed as a 
> positive outcome, unless we are talking about, say, North Korea. Silo 
> broadening can often be positive (mainly due to economies of scale), 
> but it runs the risk of scope creep and excess internal complexity and 
> interdependency (hence divestitures - which are a form of silo 
> narrowing).  Silo narrowing can also be positive when it fosters 
> greater awareness of external entities and flexibility and dynamism in 
> partnering with other silos (instead of doing everything inside the 
> silo organizational boundaries), but it runs the risk of focusing on 
> an ecological niche that might change in significant ways or evaporate
completely.
> 
> I think it is more productive to talk about reviewing/adjusting the 
> "open-ness" of silos and making them consider explicitly what should 
> be made accessible external to the silo and what the silo should try 
> to obtain from outside the silo. One should also ask how much 
> continuous/periodic monitoring of the silo's environment for changes 
> that present threats or opportunities for the silo (and its degree of 
> "open-ness") might be appropriate, given the silo's core objective(s) and
resource constraints.
> The answers to these questions would then drive the information models 
> used by the silos for internal operations and for interaction with 
> their environment. It also will drive the degree of dynamism 
> appropriate in information model strategy. Unused information model 
> flexibility can be viewed as wasted development/acquisition and run-time
resources.
> Insufficient flexibility can lead to institutional 
> failure/dissolution. Best to encourage for a "Goldilocks" approach 
> rather than saying that more open-ness or more flexibility is always
better.
> 
> Hans
> 
> -----Original Message-----
> From: ontolog-forum-bounces@xxxxxxxxxxxxxxxx
> [mailto:ontolog-forum-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Paul 
> Tyson
> Sent: Sunday, September 15, 2013 9:11 AM
> To: [ontolog-forum]
> Subject: Re: [ontolog-forum] ONTOLOG community event planning and 
> scheduling session - Thu 2013.09.12 & Thu 2013.09.19
> 
> I like Kingsley's "data-de-silo-fication" theme. (In fact, I'm soon to 
> give an internal tech talk called "Down With Silos! How linked data is 
> beautifying the information landscape").
> 
> I want to contribute a different narrative, orthogonal to the 
> engineering discussion in this thread, but leading I think to the same 
> place Kingsley is heading. For brevity I'll keep it to bullet points.
> 
> 1. Enterprises depend for their success on people in the enterprise 
> doing the right thing at the right time.
> 2. People only know what is the right thing and how to do it by 
> getting good information in the form most useful to them at the time they
need it.
> 3. They get the information they need primarily from "documents", 
> taken in the very general sense as some bounded, structured, 
> purposeful package of distinctions (glyphs, lines, colors, shapes,
texture, sound, image, etc.).
> 4. Documents, being packages of differences, can be decomposed into 
> particles of significance related in particular ways.
> 5. RDF is a good way to represent, record, and exchange particles of 
> significance that are related in particular ways. Along with XML, 
> HTML, HTTP, and related W3C standards, we have a complete suite of 
> tools for delivering documents containing the information needed to 
> the people who need it to act for the success of the enterprise.
> 
> There should be no dispute about RDBMS as an efficient storage and 
> retrieval machinery for relational data. I appreciate hearing about 
> the engineering and theoretical issues about such systems. However, 
> those issues are related to the problems of getting information to 
> people at the point of need only to the extent that system designers 
> choose to couple data persistence components to data delivery mechanisms.
One of the hallmarks of "legacy"
> systems is the unfortunate choice to closely couple these components.
> 
> I expect the discussion in this forum to focus on how to deliver 
> information to a human in the way that best meets his or her 
> constantly changing and not entirely predictable needs. Whether the 
> data is persisted on disk or papyrus, in Elbonian, SQL, NoSQL, or 
> Linear B, may be of great concern to the designers and engineers 
> tasked with supporting the information needs of an enterprise. But it 
> should be immaterial to discussions about what happens directly on 
> each side of the computer screen: that is, how  documents are composed 
> for display, and how they are interpreted by the human on the other side
of the screen.
> 
> Those of us who focus on the 2-sides-of-the-screen problem domain have 
> found the W3C basic and semantic web technology stacks of inestimable
value.
> 
> Regards,
> --Paul
> 
> On Fri, 2013-09-13 at 10:07 -0400, Kingsley Idehen wrote: 
> > On 9/13/13 9:33 AM, Michael Brunnbauer wrote:
> > 
> > > Hello Kingsley,
> > > 
> > > On Fri, Sep 13, 2013 at 08:37:01AM -0400, Kingsley Idehen wrote:
> > > > > I agree wholeheartedly. RDF and SPARQL make data integration 
> > > > > easier (without solving the fundamental issues of course).
> > > > What is the fundamental issue, as you see it?
> > > http://en.wikipedia.org/wiki/Heterogeneous_database_system#Problem
> > > s_ of_heterogeneous_database_integration
> > ## In Turtle, for sake of clarity re, my world-view ##
> > 
> > <http://en.wikipedia.org/wiki/Heterogeneous_database_system#Problems
> > _o f_heterogeneous_database_integration>
> > <#myLabel> "Data-de-silo-fication" ; <#sameAs> 
> > <#HeterogeneousDataFederation>, <#DataVirtualization>, 
> > <#DataSpaces>, <#MasterDataManagement> ; <#comment> """This problem 
> > covers data disparity issues that include:
> > shape, location, and relation semantics (or lack thereof)""" .  
> > 
> > ## Turtle End ##
> > 
> > So I assume we are in agreement re., the problem? 
> > 
> > > 
> > > http://lists.w3.org/Archives/Public/public-lod/2013Jun/0458.html
> > > 
> > > > I see the fundamental issue (or pain point) being
> data-de-silo-fication.
> > > RDF is nice for Extract Transform Load. The problems start if you 
> > > want to change data.
> > 
> > Change sensitivity is handled via the use of Linked Data Views over 
> > disparate data sources. This is what R2RML facilitates albeit rarely 
> > mentioned, sadly.
> > 
> > Views can be transient, materialized, or a configurable mix of both.
> > That's certainly the case re. Virtuoso i.e., make a change in its 
> > SQL DBMS (or a remote ODBC or JDBC accessible DBMS) and they are 
> > reflected in all your SPARQL queries and Linked Data URI lookups. 
> > The same even applies to RESTful or SOA services that are attached 
> > to Virtuoso (we cover 100+ protocols and formats).
> > 
> > We have Replication (Snapshot and Transactional)  and HTTP 
> > (including cache invalidation) baked into Virtuoso.
> > 
> > > 
> > > > > But they are a bad option for data storage because maintaining 
> > > > > consistency is so difficult (think about deleting a row or 
> > > > > transactions).
> > > > I don't know what that really means.
> > > Suppose you have an App with user registration. If you store the 
> > > user data in a triple store, deleting a user with SPARQL becomes
> difficult.
> > 
> > That doesn't apply to every triplestore. That doesn't apply to 
> > Virtuoso. We even have large customer running OLTP like workflows 
> > with something like 40 million named graphs. BTW -- as part of the 
> > workflow,  Virtuoso has to factor in deltas such that it doesn't 
> > perform wholesale named graph deletions etc.
> > 
> > > Removing
> > > a single triple is not enough. Storing the user in a named graph 
> > > may help but probably creates other problems and definitely makes 
> > > querying a lot more complicated.
> > > 
> > > What about SPARQL transactions ? Starting a transaction, reading 
> > > and updating, commiting the transaction.
> > 
> > We are a full blown ACID DBMS. See our benchmark reports. These 
> > simply aren't new issues since we have a hybrid DBMS.
> > 
> > > Is there a triple store that supports this with all the fidelity 
> > > of modern RDB systems ?
> > 
> > Yes. It's called Virtuoso !
> > 
> > > 
> > > > I say that because we simply don't have that problem in our 
> > > > hybrid
> DBMS.
> > > I don't know what that really means. Can I modify data with SPARQL
> > > *and* SQL in your DBMS ? If yes, how does that work ?
> > 
> > Of course you can. We support SPARQL 1.1 Update. We are SQL-99 
> > compliant. We do ACID. We have serious customers doing OLTP like 
> > stuff using RDF or SQL aspects of Virtuoso. [1][2][3][4]
> > 
> > Links:
> > 
> > 1. http://bit.ly/ZOCmaD -- shows we even have the performance 
> > difference between SPARQL and SQL down to insignificant levels via 
> > Star Schema Benchmark Report 2. http://bit.ly/10pvAbF -- blog post 
> > about this effort 3. http://bit.ly/Yf5etP -- Berlin SPARQL Benchmark 
> > Report (note: this particular benchmark is SQL relational DBMS
> > oriented) 4. http://bit.ly/14ULX2F -- 150 Billion triples scale 
> > report 5. http://bit.ly/RtdGjA -- CoRelational DBMS Concepts post 
> > that includes live links to R2RML Views built atop SQL data 6.
> > http://bit.ly/13fnIbr -- example of R2RML views atop an Oracle DBMS 
> > hooked into Virtuoso via ODBC .
> > 
> > 
> > Kingsley
> > > 
> > > Regards,
> > > 
> > > Michael Brunnbauer
> > > 
> > > 
> > > 
> > > _________________________________________________________________
> > > Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
> > > Config Subscr: 
> > > http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
> > > Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
> > > Shared Files: http://ontolog.cim3.net/file/ Community Wiki: 
> > > http://ontolog.cim3.net/wiki/ To join: 
> > > http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
> > >  
> > 
> > 
> > --
> > 
> > Regards,
> > 
> > Kingsley Idehen           
> > Founder & CEO 
> > OpenLink Software     
> > Company Web: http://www.openlinksw.com Personal Weblog: 
> > http://www.openlinksw.com/blog/~kidehen
> > Twitter/Identi.ca handle: @kidehen
> > Google+ Profile: https://plus.google.com/112399767740508618350/about
> > LinkedIn Profile: http://www.linkedin.com/in/kidehen
> > 
> > 
> > 
> > 
> > _________________________________________________________________
> > Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
> > Config Subscr: 
> > http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
> > Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
> > Shared Files: http://ontolog.cim3.net/file/ Community Wiki: 
> > http://ontolog.cim3.net/wiki/ To join: 
> > http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
> >  
> 
>  
> _________________________________________________________________
> Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
> Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
> Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
> Shared Files: http://ontolog.cim3.net/file/ Community Wiki:
> http://ontolog.cim3.net/wiki/ To join:
> http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
>  
> 
> 
> _________________________________________________________________
> Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
> Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
> Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
> Shared Files: http://ontolog.cim3.net/file/ Community Wiki: 
> http://ontolog.cim3.net/wiki/ To join: 
> http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
>      (014)


_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/ Community Wiki:
http://ontolog.cim3.net/wiki/ To join:
http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J    (015)



_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/  
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/  
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/ 
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J    (016)

<Prev in Thread] Current Thread [Next in Thread>
  • [ontolog-forum] Data, Silos, Interoperability, and Agility, Hans Polzer <=