oor-users
[Top] [All Lists]

Re: [oor-users] Fwd: new ontology stuck(?) behind parse queue

To: OOR Users discussion <oor-users@xxxxxxxxxxxxxxxx>
From: Paul R Alexander <palexander@xxxxxxxxxxxx>
Date: Fri, 10 Dec 2010 15:24:19 -0800
Message-id: <4D02B6A3.9090907@xxxxxxxxxxxx>
(a) Parsing is done serially on a per-load scheduler run. The load scheduler checks for new, unparsed ontologies whenever it is run (we run it every hour, I'm not sure how OOR is configured). Nothing should ever get stuck.

It looks like maybe the server was rebooted in the middle of a load scheduler run or something, which then caused these 02/12/2010 ontologies to remain in "parsing" status. This status would mean that they were never processed again. You may be able to use the admin tool to trigger a re-parse or go into the ncbo_ontology_load_queue table in the database and manually set the status column to 1 for these ontologies.

(b) If it's a large ontology (500,000+ terms) it could take 6-8 hours to process, but anything up to 100,000 terms should process in under an hour, usually much faster. Again, nothing should ever be in "parsing" status for too long, it should error out at some point and get the "error" status set.

(c) I would give up after 24 hours or less depending on the ontology size.

Finally, please keep in mind that the UI does aggressive caching. We don't expect the system to respond in real-time to many events and as such will often cache results for 4-8 hours. For example, the ontology uploaded today has actually finished parsing, it's just that the UI hasn't refreshed the information.
http://oor-01.cim3.net:8080/bioportal/ontologies/10187 shows statusId 3 ("ready")
http://oor-01.cim3.net/visualize/10187

Cache timeouts can be modified in this file on the UI:
/app/models/data_access.rb

Paul R Alexander
Web / UI Developer
NCBO BioPortal
Stanford Center for Biomedical Informatics Research


On 12/10/10 3:00 PM, Peter Yim wrote:
Thank you for the input, Bob. ...

Let find out from our friends on the BioPortal team (whose technology
we are employing in the current OOR-sandbox instance),

(a)  on whether "parsing" is queued and done serially, or whether they
are dealt with in parallel (so that ontologies with errors that are
"stuck" in "parsing" aren't block subsequent uploads.)

(b)  for recommendations on how one should deal with those they are
"stuck" at "Parsing Ontology" for a long time.

(c)  .... and, by the way, when should one consider giving up?2 days?
14 days? 30 days? ... we probably all know that those from 02/12/2010
have little chance of making it to the finish. :-)

... Misha? ( ... Paul? ... Natasha?)

Thanks & regards.  =ppy
--


---------- Forwarded message ----------
From: Bob Morris <morris.bob@xxxxxxxxx>
Date: Fri, Dec 10, 2010 at 2:27 PM
Subject: [oor-users] new ontology stuck(?) behind parse queue
To: oor-users@xxxxxxxxxxxxxxxx


Around 4 hours ago, I uploaded a new ontology to the sandbox, whose
status remains "Waiting to parse". Apparently ahead of it are about 10
ontologies uploaded 02/12/2010 (sic!)  whose status is "Parsing
Ontology".  Are those keeping mine from getting parsed, and if so,
what should I do about it?

Thanks
Bob Morris


--
Robert A. Morris
Emeritus Professor  of Computer Science
UMASS-Boston
100 Morrissey Blvd
Boston, MA 02125-3390
Associate, Harvard University Herbaria
email: morris.bob@xxxxxxxxx
web: http://bdei.cs.umb.edu/
web: http://etaxonomy.org/mw/FilteredPush
http://www.cs.umb.edu/~ram
phone (+1) 857 222 7992 (mobile)

_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/oor-users/
Subscribe: mailto:oor-users-join@xxxxxxxxxxxxxxxx
Config/Unsubscribe: http://ontolog.cim3.net/mailman/listinfo/oor-users/
Shared Files: http://ontolog.cim3.net/file/work/OpenOntologyRepository/
Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OpenOntologyRepository
 
_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/oor-users/  
Subscribe: mailto:oor-users-join@xxxxxxxxxxxxxxxx 
Config/Unsubscribe: http://ontolog.cim3.net/mailman/listinfo/oor-users/ 
Shared Files: http://ontolog.cim3.net/file/work/OpenOntologyRepository/  
Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OpenOntologyRepository 
 

_________________________________________________________________
Message Archives: http://ontolog.cim3.net/forum/oor-users/  
Subscribe: mailto:oor-users-join@xxxxxxxxxxxxxxxx 
Config/Unsubscribe: http://ontolog.cim3.net/mailman/listinfo/oor-users/ 
Shared Files: http://ontolog.cim3.net/file/work/OpenOntologyRepository/  
Wiki: http://ontolog.cim3.net/cgi-bin/wiki.pl?OpenOntologyRepository     (01)

<Prev in Thread] Current Thread [Next in Thread>