[Top] [All Lists]

Re: [ontolog-forum] independent semantic software evaluation frameworks?

To: AzamatAbdoullaev <abdoul@xxxxxxxxxxxxxx>
Cc: semantic-web@xxxxxxxxxxxxxxxxxx, "[ontolog-forum]" <ontolog-forum@xxxxxxxxxxxxxxxx>
From: Paola Di Maio <paola.dimaio@xxxxxxxxx>
Date: Mon, 10 Aug 2009 16:22:23 +0100
Message-id: <4a4804720908100822w3f25db04r24b3c054f4b7848d@xxxxxxxxxxxxxx>

On Mon, Aug 10, 2009 at 3:07 PM, AzamatAbdoullaev <abdoul@xxxxxxxxxxxxxx> wrote:
PD wrote: "We are particularly interested initially in the software developed with EU funding (FP6 and FP7 )
You are doing a great work for the public, seemingly without any substantial compensation.

Doing interesting and important work in freedom is the highest possible compensation for an inquirying mind and as you know, some things are priceless anyway.
Humanity ows a lot to scores of unrewarded people who advance the progress of humanity with personal sacrifices, often unrecognised, and sometimes even receiving endless aggravation...

(I watched a movie recently on the history of pennicillin, blimey, a simple fungus made such a difference and the luminaries would not want to invest a penny on it for so long
As i know, you got this assignment from the EC Commissioner of Information Society&Media. And as i understood, almost nobody, who got the funds through this department, responded to your good initiative. But there are at least 35 research groups enlisted there, as below:

They are not supposed to. I am expecing that the wider community, the 'public' evaluates the output of the research.
For some reasons there is no open respository for such evaluations, so I am pulling some together.

I am admittedly rather limited dexterity with tools and need a lot of handholding, (I always ask myself if something is  really not working, or is it me ?)

Ideally it's evaluators with different profiles (from totally inept to very apt). At the least we'll come up with some suggestions on how to make tools easier to work for the rest of us.

Wonder what are you going to report?

Chapters headings and evaluation criteria will be circulated to those who express interest in chipping in..

But if you need the outlines, i can share the summaries done for an international conference.

We all have our personal reasons to like or not like certain research, projects, software. What I am trying to do here is to establish some objective parameters to ground discussions on and to find a way to include public feedback and wider participation in the research funding process,  It's not going to be easy, but when tools claim to be produced for the public good (open source? semantic, social software?) then there must be mechanism for the public to pitch in.

The public has the right to transparency and accountability of publicly funded research. If not, maybe there is more work to be done at policy level. Whatever. So send in word or pdf any materials you think can support the above, the rest it will be a question of working things out.  As usual.

Who has got ears will listen....



----- Original Message -----

Sent: Monday, August 10, 2009 1:00 PM
Subject: Re: independent semantic software evaluation frameworks?

Thanks to all for replies!
see notes below

There are a few that I am personally trying to run, with varying degrees of success, but any tool that people want to review from the pool below, for example. Will circulate evaluation sheet to those who want do run something and contact me offlist pls.  At this stage we prioritize publicly funded ones, rather than commercial tools.




I am not adding editors to any list, and again, prioritizing publicly funded. i dont think your tool is?

Yes, thanks! will follow up and ....

"In any case the problem is rather social/cultural/organisational than merely technical." 

so true... for some of us, technology is mainly a mean to solve real world problems in real world context, so any technical issue is, at least in part, determined by social/cultural/organisational issues. This is often forgotten, and technology is done for technology's sake.

Thanks a lot for spreading the word
Even more value would come from this being a public , open, collaborative effort......

I am still working on the evaluation sheet, would welcome thoughts and criticism, so please get in touch  if you are interested



On Sun, Aug 9, 2009 at 6:03 PM, Paola Di Maio<
paola.dimaio@xxxxxxxxx> wrote:
> Greetings good people!
> I am attempting to systematically aggregate as many independent
> evaluations of semantic software tools as possible
> (Note: by independent it is intended not carried out by people who are
> paid to do so, or have other vested interests  - such as the
> devleopers or consultants themselves  or their friends and familieis -
>  but by unaffiliated users who are more suitable to appreciate its
> benefits and shorcomings)
> We are particularly interested initially in the software developed
> with EU funding (FP6 and FP7 )
> The main frameworks of reference are standard software project
> evaluation methods, including
> references included there)
> Md. Ahsan-ul Murshed and Ramanjit Singh
> I have already contacted offlist some of the individuals who have
> offered knowledge and views, and I am now  seeking additional wider
> general  input on
> 1.  developing and customizing the  review criteria  and methodology
> (so that we can format the distributed evaluations using a common template)
> 2.  any volunteered inputs in terms of reviews of software and other
> deliverables (priority is to assess  cost/benefit ratio, measured in
> terms of
> functionality and usefulness)
> The final goal of this exercise is to contribute  to improve the
> effectiveness of EU funding process in semantic web, as well as to
> general
> software output at large, as well as to come up with  recommendations.
> Your contribution can be submitted authored or anonymous (subject to
> verification)
> Please contact me offlist should you wish to collaborate on this project,
> cheers
> Paola Di Maio
> --
> Paola Di Maio
> **************************************************
> Strategic Advisor
> Networked Research Lab, UK
> **************************************************

Paola Di Maio
DMEM (Design, Manufacture, Engineering, Management)
University of  Strathclyde
Room 106, 75 Montrose Street
Glasgow G1 1XJ UK

Paola Di Maio
Networked Research Lab, UK


Message Archives: http://ontolog.cim3.net/forum/ontolog-forum/  
Config Subscr: http://ontolog.cim3.net/mailman/listinfo/ontolog-forum/  
Unsubscribe: mailto:ontolog-forum-leave@xxxxxxxxxxxxxxxx
Shared Files: http://ontolog.cim3.net/file/
Community Wiki: http://ontolog.cim3.net/wiki/ 
To join: http://ontolog.cim3.net/cgi-bin/wiki.pl?WikiHomePage#nid1J
To Post: mailto:ontolog-forum@xxxxxxxxxxxxxxxx    (01)

<Prev in Thread] Current Thread [Next in Thread>