Empirical is a platform that allows scientists to run, share and compare experiments, including their whole research environment.
WHY EMPIRICAL?
The idea for this product comes from the frustrations that I had while doing research in computer vision. There are many inefficiencies slowing down modern science, from trying to figure out what’s the state of the art to the lack of easy means to reproduce and evaluate research.
We aim to attack these inefficiencies by providing a standard framework to run, share and compare experiments. Bringing some of the best practices in software development to science and making it easy for researchers to adopt them. Also with each experiment that is performed in the platform we’re hoping to build a database to map the state of the art.
We know this is idea will make a big impact in the way science is made. This has the potential to change the meaning of peer review, and improve reproducibility and discovery, which will lead to faster iteration cycles and better science.
STATUS:
We’re currently working through the prototype and hoping to have an alpha version soon. Meanwhile, you can stay updated by signing up on our site: http://empiricalci.com
BUSINESS MODEL:
SaaS, paid private plans + paid storage and computing time.
ABOUT US:
We worked together for over a year in a different startup. Bren as a UX product designer and Alan as a computer vision engineer and tech lead.
Before that, Bren studied CS at Berkeley and then specialized in UX at General Assembly. Alan was doing a Ph.D in AI systems, specializing in Computer Vision at Monterrey Institute of Technology.
Feedback is greatly appreciated and please let me know any question you may have.
Down the road we'll be able to add support for specialized lab equipment in a case by cases basis.
For Windows only software, Docker and Microsoft seem to be making progress on that front. It's still to be defined how it's going to play out.
Fortunately there are many cross-platform alternatives for most scientific software (at least in the AI area). Curious to know, do you have a particular software in mind?
No. I was comparing the requirements against my ability to imagine messy details. I suppose Docker jumped out as a different level of abstraction relative to the general problems of scientific reproducibility; the diversity of scientific investigations; and the existing social and physical infrastructure of scientific laboratories.
I suppose the sort of messy detail I might imagine is how does a Matlab license work in terms of multiple containers. And the details are messy because of legal and institutional constraints in the markets I suspect are looking for this solution.
Yes, MATLAB is a special case due to it being licensed software.
I've used it in the past by mounting my local MATLAB installation into the container and I think I had to pass the MAC too.
I believe we'll be able to get it to the point where we only need to mount the local license.
We're building a client on top of docker to deal with all this extra functionality. You can think about it like a specialized version of docker-compose
This is pretty interesting. Are you asking users to place code on your platform? So a user runs a test in Matlab and gets a certain set of results, you are asking them to upload the code and the results so that others can see what has been done and what the results are?
That's the intended result. Although, we're not hosting the code, we're integrating with GitHub for that. We're going to be hosting the Docker images and the results/output.
You'll be able to see which version of the code produces certain results.
Empirical is a platform that allows scientists to run, share and compare experiments, including their whole research environment.
WHY EMPIRICAL? The idea for this product comes from the frustrations that I had while doing research in computer vision. There are many inefficiencies slowing down modern science, from trying to figure out what’s the state of the art to the lack of easy means to reproduce and evaluate research. We aim to attack these inefficiencies by providing a standard framework to run, share and compare experiments. Bringing some of the best practices in software development to science and making it easy for researchers to adopt them. Also with each experiment that is performed in the platform we’re hoping to build a database to map the state of the art.
We know this is idea will make a big impact in the way science is made. This has the potential to change the meaning of peer review, and improve reproducibility and discovery, which will lead to faster iteration cycles and better science.
STATUS: We’re currently working through the prototype and hoping to have an alpha version soon. Meanwhile, you can stay updated by signing up on our site: http://empiricalci.com
BUSINESS MODEL: SaaS, paid private plans + paid storage and computing time.
ABOUT US: We worked together for over a year in a different startup. Bren as a UX product designer and Alan as a computer vision engineer and tech lead. Before that, Bren studied CS at Berkeley and then specialized in UX at General Assembly. Alan was doing a Ph.D in AI systems, specializing in Computer Vision at Monterrey Institute of Technology.
Feedback is greatly appreciated and please let me know any question you may have.
It appears to require Docker. How will this play out with Windows only software and programmable lab equipment?
Down the road we'll be able to add support for specialized lab equipment in a case by cases basis.
For Windows only software, Docker and Microsoft seem to be making progress on that front. It's still to be defined how it's going to play out. Fortunately there are many cross-platform alternatives for most scientific software (at least in the AI area). Curious to know, do you have a particular software in mind?
No. I was comparing the requirements against my ability to imagine messy details. I suppose Docker jumped out as a different level of abstraction relative to the general problems of scientific reproducibility; the diversity of scientific investigations; and the existing social and physical infrastructure of scientific laboratories.
I suppose the sort of messy detail I might imagine is how does a Matlab license work in terms of multiple containers. And the details are messy because of legal and institutional constraints in the markets I suspect are looking for this solution.
Yes, MATLAB is a special case due to it being licensed software. I've used it in the past by mounting my local MATLAB installation into the container and I think I had to pass the MAC too. I believe we'll be able to get it to the point where we only need to mount the local license.
We're building a client on top of docker to deal with all this extra functionality. You can think about it like a specialized version of docker-compose
This is pretty interesting. Are you asking users to place code on your platform? So a user runs a test in Matlab and gets a certain set of results, you are asking them to upload the code and the results so that others can see what has been done and what the results are?
That's the intended result. Although, we're not hosting the code, we're integrating with GitHub for that. We're going to be hosting the Docker images and the results/output. You'll be able to see which version of the code produces certain results.