[Developers] Software development process
Matthew Supernaw
matthew.supernaw at noaa.gov
Thu Apr 5 08:05:00 PDT 2012
All-
Thanks for the comments. Let me start by making it clear that the
intention of my post was not to change anything in particular, but
rather, just put the admb development process into an official
document. A formal plan would serves as the basis for managing and
tracking the groups overall efforts. In addition, it can be
particularly helpful for new developers coming on board since the SDP
would outline objectives, methodology, coding standards, tools, etc.
We could also use a SDP to addresses issues like bug fixes,
prototyping, modernization, verification, and validation.
Johnoel- Great! I have some experience with iterative development,
however mostly waterfall. Agile sounds like a great methodology to me.
An official SDP would help me fulfill my obligations to NOAA Fisheries
and the NOAA Fisheries Assessment Steering Committee. I would like to
continue this conversation offline. Please let me know your
availability.
Arni- There is no doubt, Johnoel is doing a great job! Your assessment
is correct. I realize that not everyone in the group is a professional
software developer or even a full-time contributor. I'm simply trying
to encourage admb developers to become more familiar with large-scale
software project development/management. An official SDP would help
streamline a lot of the process, as well as, help answer a lot of
questions that arise during development. BTW, I would never want to
take the fun out of writing code.
Dave- Software development plans are common practice, to include
open-source. http://gcc.gnu.org/develop.html
Matthew
On Wed, Apr 4, 2012 at 4:26 PM, <developers-request at admb-project.org> wrote:
>
> Send Developers mailing list submissions to
> developers at admb-project.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.admb-project.org/mailman/listinfo/developers
> or, via email, send a message with subject or body 'help' to
> developers-request at admb-project.org
>
> You can reach the person managing the list at
> developers-owner at admb-project.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Developers digest..."
>
>
> Today's Topics:
>
> 1. Re: Software development process (Arni Magnusson)
> 2. Re: Software development process (Johnoel Ancheta)
> 3. Re: Software development process (dave fournier)
> 4. Re: Software development process (Arni Magnusson)
> 5. Re: Check Release procedure (dave fournier)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 4 Apr 2012 20:08:37 +0000 (GMT)
> From: Arni Magnusson <arnima at hafro.is>
> To: developers at admb-project.org
> Subject: Re: [Developers] Software development process
> Message-ID: <alpine.LFD.2.02.1204041836220.6630 at hafstokkur.hafro.is>
> Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
>
> Thanks, Matthew, for stirring our minds.
>
> As the ADMB Project grows (number of people, number of features, number of
> bugs, etc.), this may be a good time to think about whether we can benefit
> from adopting certain software development methodologies.
>
> You and Johnoel know more about this than the others. It sounds like you
> are not suggesting any specific methodologies at this point, just
> encouraging everyone to read and think about them.
>
> ---
>
> Generally, the ADMB Project follows the methodology known as
>
> http://en.wikipedia.org/wiki/Cowboy_coding
>
> The main advantages are light overhead and fun. As most of us contribute
> to ADMB in our free time, it must be fun. Cowboy coding is at one extreme,
> and at the other extreme we have Dilbert coding, with overly complex and
> vague methodologies that keep changing. The ADMB Project is moving only
> slightly towards the middle ground. Dilbert in chaps?
>
> Automated testing (Buildbot and unit tests) is a good example, where
> Johnoel has radically improved the development process - without adding
> any burden or requirements on the developers.
>
> The annual developers' workshop is also a key "methodology", where we make
> sure things are moving in a sensible direction, in a sensible way. They
> also mark the beginning of the release procedure, generate new ideas, and
> add to the fun factor.
>
> Version control is a given, and SVN is what cowboys ride. Actually, I have
> one suggestion related to ADMB development methodology: to install ViewVC,
> so we can browse and diff the repository using a web browser. My sysadmin
> has deployed it here at Hafro with success. This is what it looks like:
>
> http://gcc.gnu.org/viewcvs/trunk/
> http://r-forge.r-project.org/scm/viewvc.php/pkg/?root=glmmadmb
>
> Thinking about potentially useful development methods/tools doesn't hurt.
> My feeling is that the ADMB Project is doing reasonably well, with
> developers enthusiastically contributing to different aspects of the
> project. Johnoel serves a key role, including quality control, and is
> doing a great job.
>
> Arni
>
>
>
> On Wed, 4 Apr 2012, Matthew Supernaw wrote:
>
> > All,
> >
> > Please checkout the follow link:
> >
> > http://en.m.wikipedia.org/wiki/Software_development_process
> >
> > I recommend that we develop a formal and concise software development
> > plan, it will make our work much easier!
> >
> > Matthew
> >
>
>
> ------------------------------
>
> Message: 2
> Date: Wed, 04 Apr 2012 10:11:51 -1000
> From: Johnoel Ancheta <johnoel at hawaii.edu>
> To: Arni Magnusson <arnima at hafro.is>
> Cc: developers at admb-project.org
> Subject: Re: [Developers] Software development process
> Message-ID:
> <CAJMx2XXR4AdcpiORgks0OCwNMar-mRdehrS-PMRaHTnQGLbk-g at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Redmine includes a ViewVC like service
>
> http://www.admb-project.org/redmine/projects/issues/repository
>
>
> On Wed, Apr 4, 2012 at 10:08 AM, Arni Magnusson <arnima at hafro.is> wrote:
>
> > Thanks, Matthew, for stirring our minds.
> >
> > As the ADMB Project grows (number of people, number of features, number of
> > bugs, etc.), this may be a good time to think about whether we can benefit
> > from adopting certain software development methodologies.
> >
> > You and Johnoel know more about this than the others. It sounds like you
> > are not suggesting any specific methodologies at this point, just
> > encouraging everyone to read and think about them.
> >
> > ---
> >
> > Generally, the ADMB Project follows the methodology known as
> >
> > http://en.wikipedia.org/wiki/**Cowboy_coding<http://en.wikipedia.org/wiki/Cowboy_coding>
> >
> > The main advantages are light overhead and fun. As most of us contribute
> > to ADMB in our free time, it must be fun. Cowboy coding is at one extreme,
> > and at the other extreme we have Dilbert coding, with overly complex and
> > vague methodologies that keep changing. The ADMB Project is moving only
> > slightly towards the middle ground. Dilbert in chaps?
> >
> > Automated testing (Buildbot and unit tests) is a good example, where
> > Johnoel has radically improved the development process - without adding any
> > burden or requirements on the developers.
> >
> > The annual developers' workshop is also a key "methodology", where we make
> > sure things are moving in a sensible direction, in a sensible way. They
> > also mark the beginning of the release procedure, generate new ideas, and
> > add to the fun factor.
> >
> > Version control is a given, and SVN is what cowboys ride. Actually, I have
> > one suggestion related to ADMB development methodology: to install ViewVC,
> > so we can browse and diff the repository using a web browser. My sysadmin
> > has deployed it here at Hafro with success. This is what it looks like:
> >
> > http://gcc.gnu.org/viewcvs/**trunk/ <http://gcc.gnu.org/viewcvs/trunk/>
> > http://r-forge.r-project.org/**scm/viewvc.php/pkg/?root=**glmmadmb<http://r-forge.r-project.org/scm/viewvc.php/pkg/?root=glmmadmb>
> >
> > Thinking about potentially useful development methods/tools doesn't hurt.
> > My feeling is that the ADMB Project is doing reasonably well, with
> > developers enthusiastically contributing to different aspects of the
> > project. Johnoel serves a key role, including quality control, and is doing
> > a great job.
> >
> > Arni
> >
> >
> >
> >
> > On Wed, 4 Apr 2012, Matthew Supernaw wrote:
> >
> > All,
> >>
> >> Please checkout the follow link:
> >>
> >> http://en.m.wikipedia.org/**wiki/Software_development_**process<http://en.m.wikipedia.org/wiki/Software_development_process>
> >>
> >> I recommend that we develop a formal and concise software development
> >> plan, it will make our work much easier!
> >>
> >> Matthew
> >>
> >> ______________________________**_________________
> > Developers mailing list
> > Developers at admb-project.org
> > http://lists.admb-project.org/**mailman/listinfo/developers<http://lists.admb-project.org/mailman/listinfo/developers>
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.admb-project.org/pipermail/developers/attachments/20120404/73f66336/attachment-0001.html>
>
> ------------------------------
>
> Message: 3
> Date: Wed, 04 Apr 2012 13:13:18 -0700
> From: dave fournier <davef at otter-rsch.com>
> To: developers at admb-project.org
> Subject: Re: [Developers] Software development process
> Message-ID: <4F7CAB5E.5000808 at otter-rsch.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 12-04-04 01:08 PM, Arni Magnusson wrote:
>
> I've hard all this bullshit so many times before. You take a bunch of
> people who don't know what they are doing and form a committee. The you
> tell the committee to formulate a complete design plan.
> Six months later you put it in a nice document. A year later you start
> over. If open source needs this
> maybe that is what's wrong with open source.
>
>
>
> > Thanks, Matthew, for stirring our minds.
> >
> > As the ADMB Project grows (number of people, number of features,
> > number of bugs, etc.), this may be a good time to think about whether
> > we can benefit from adopting certain software development methodologies.
> >
> > You and Johnoel know more about this than the others. It sounds like
> > you are not suggesting any specific methodologies at this point, just
> > encouraging everyone to read and think about them.
> >
> > ---
> >
> > Generally, the ADMB Project follows the methodology known as
> >
> > http://en.wikipedia.org/wiki/Cowboy_coding
> >
> > The main advantages are light overhead and fun. As most of us
> > contribute to ADMB in our free time, it must be fun. Cowboy coding is
> > at one extreme, and at the other extreme we have Dilbert coding, with
> > overly complex and vague methodologies that keep changing. The ADMB
> > Project is moving only slightly towards the middle ground. Dilbert in
> > chaps?
> >
> > Automated testing (Buildbot and unit tests) is a good example, where
> > Johnoel has radically improved the development process - without
> > adding any burden or requirements on the developers.
> >
> > The annual developers' workshop is also a key "methodology", where we
> > make sure things are moving in a sensible direction, in a sensible
> > way. They also mark the beginning of the release procedure, generate
> > new ideas, and add to the fun factor.
> >
> > Version control is a given, and SVN is what cowboys ride. Actually, I
> > have one suggestion related to ADMB development methodology: to
> > install ViewVC, so we can browse and diff the repository using a web
> > browser. My sysadmin has deployed it here at Hafro with success. This
> > is what it looks like:
> >
> > http://gcc.gnu.org/viewcvs/trunk/
> > http://r-forge.r-project.org/scm/viewvc.php/pkg/?root=glmmadmb
> >
> > Thinking about potentially useful development methods/tools doesn't
> > hurt. My feeling is that the ADMB Project is doing reasonably well,
> > with developers enthusiastically contributing to different aspects of
> > the project. Johnoel serves a key role, including quality control, and
> > is doing a great job.
> >
> > Arni
> >
> >
> >
> > On Wed, 4 Apr 2012, Matthew Supernaw wrote:
> >
> >> All,
> >>
> >> Please checkout the follow link:
> >>
> >> http://en.m.wikipedia.org/wiki/Software_development_process
> >>
> >> I recommend that we develop a formal and concise software development
> >> plan, it will make our work much easier!
> >>
> >> Matthew
> >>
> > _______________________________________________
> > Developers mailing list
> > Developers at admb-project.org
> > http://lists.admb-project.org/mailman/listinfo/developers
> >
>
>
>
> ------------------------------
>
> Message: 4
> Date: Wed, 4 Apr 2012 20:23:06 +0000 (GMT)
> From: Arni Magnusson <arnima at hafro.is>
> To: developers at admb-project.org
> Subject: Re: [Developers] Software development process
> Message-ID: <alpine.LFD.2.02.1204042015300.6630 at hafstokkur.hafro.is>
> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
>
> Damn straight, partner!
>
> Handpick a few good cowboys and light a fire. Then fill a pot with beans
> and shoot the cans. Brand a few bulls. Make, verify, and sleep under a
> tree.
>
>
>
>
> On Wed, 4 Apr 2012, dave fournier wrote:
>
> > On 12-04-04 01:08 PM, Arni Magnusson wrote:
> >
> > I've hard all this bullshit so many times before. You take a bunch of
> > people who don't know what they are doing and form a committee. The you
> > tell the committee to formulate a complete design plan. Six months later
> > you put it in a nice document. A year later you start over. If open
> > source needs this maybe that is what's wrong with open source.
> >
>
>
> ------------------------------
>
> Message: 5
> Date: Wed, 04 Apr 2012 13:26:11 -0700
> From: dave fournier <davef at otter-rsch.com>
> To: developers at admb-project.org
> Subject: Re: [Developers] Check Release procedure
> Message-ID: <4F7CAE63.70709 at otter-rsch.com>
> Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"
>
> On 12-04-04 09:55 AM, Ian Taylor wrote:
>
> I pretty sure that you (almost) all think I'm unreasonable. I've been
> told (too many times) that R was designed by a bunch of geniuses. Here
> is a typical attempt to make a trivial change to a model
> by one of the greatest heroes of open source.
>
> On Thu, Sep 30, 2010 at 3:29 PM, Douglas Bates<bates at stat.wisc.edu <https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>> wrote:
> >/ Unfortunately an nlmer model is not appropriate for a binary response,
> />/ because it doesn't appropriately weight the residuals.
> />/
> />/ Incorporating a non-zero guessing parameter requires a generalized
> />/ nonlinear mixed model if you need to estimate the guessing parameter.
> />/ The long term plan is to allow such a model. This is the reason that
> />/ Martin and I worked on factoring out the internal code dealing with
> />/ different kinds of models for the expected response. Nonlinear models
> />/ affect these in one way and generalized linear models in another so
> />/ you need to chain these effects.
> />/
> />/ For the particular case that Robert is considering, in which the
> />/ guessing parameter is fixed at 0.33 I think it may be possible to use
> />/ the mafc.logit link from the psyphy package with lme4a, the
> />/ development version of lme4. I am currently installing the necessary
> />/ packages to see if I can make it work. My thanks to Robert for making
> />/ the data available so we can test it.
> /
> It wasn't as easy as I had hoped it would be. I'm getting an error
> when evaluating the linkfun (and, presumably, will get such an error
> for all the other functions in the family). It probably has to do
> with the environment in which the function is evaluated in that it
> can't see the value of 'm'.
>
> I'm not sure if I will be able to fix it in a reasonable amount of
> time (I should be grading assignments from one of my classes right
> now).
>
> Actually the whole design of the glm families should be reconsidered
> but we'll save that for another time.
>
> >/ On Mon, Sep 27, 2010 at 11:25 AM, Manuel Morales
> />/ <Manuel.A.Morales at williams.edu <https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models>> wrote:
> />>/ I found this link doing a search for your error message on Google:
> />>/ https://stat.ethz.ch/pipermail/r-sig-mixed-models/2007q4/000408.html
> />>/
> />>/ Following the recipe:
> />>/ grModel<- function(x,y,a,b,c,d) .33 + .67*(exp(log(x)*a+y*b+log(x)*y*c
> />>/ + d))/
> />>/ (1+exp( log(x)*a + y*b +log(x)*y*c + d))
> />>/
> />>/ grModg<- deriv(body(grModel), namevec = c("a","b","c","d"),
> />>/ function.arg=grModel)
> />>/
> />>/ mod3<- nlmer(Correct~grModg(Concentrat,Test,a,b,c,d)~(Test|Code),
> />>/ start = c(a = 0.115, b=-0.1, c=0.65, d=-3),
> />>/ data = rawdata)
> />>/ Which appears to work.
> />>/
> />>/ My messages haven't been posted to R, so you may want to post again with
> />>/ this solution if it works for you.
> />>/
> />>/ Best,
> />>/
> />>/ Manuel
> />>/
> />>/ On Mon, 2010-09-27 at 15:54 +0200, Robert Miller wrote:
> />>>/ Hello everyone,
> />>>/
> />>>/ Recently i tried to predict the discrimination probability of a chemosignal
> />>>/ by its concentration and an experimental manipulation factor (term:
> />>>/ concentration*x + test*b + concentration*test*c + d) with nested factor
> />>>/ "manipulation" within "participants". For statistical analysis i needed to
> />>>/ incorporate a fixed guessing probability into my model (similiar to a 3-PL
> />>>/ IRT model) resulting in the following equation:
> />>>/
> />>>/ P(correct) = 0.33 + 0.67*(exp(term)/(1 + exp(term)))
> />>>/
> />>>/ As i found no way to do so via the glmer()-function of the lme4-package, i
> />>>/ tried to use nlmer() but unfortunately even the simplest analysis with just
> />>>/ the concentration factor and intercept resulted in cryptic error messages.
> />>>/
> />>>/ Syntax:
> />>>/ library(lme4)
> />>>/ rawdata<- read.csv2("http://dl.dropbox.com/u/7147679/AND_data.csv")
> />>>/
> />>>/ mod1<- glmer(Correct ~ log(Concentrat) * Test + (Test|Code), family =
> />>>/ binomial, data=rawdata) #works fine but is inappropriate
> />>>/ mod2<- nlmer(Correct ~ .33 + .67*(exp(log(Concentrat)*a+d))
> />>>/ /(1+exp(log(Concentrat)*a+d)) ~ (Test|Code), start = c(a = 0.1, d = -3),
> />>>/ data = rawdata) #doesnt work
> />>>/ mod3<- nlmer(Correct ~ .33 + .67*(exp(log(Concentrat)*a + Test*b +
> />>>/ log(Concentrat)*Test*c + d))/(1+exp( log(Concentrat)*a + Test*b +
> />>>/ log(Concentrat)*Test*c + d)) ~ (Test|Code), start = c(a = 0.115,b = -0.05,
> />>>/ c= 0.065, d= -3), data = rawdata) #doesnt work either
> />>>/
> />>>/ Even without specifying random effects nls() doesnt work, but brute force
> />>>/ ML-parameter estimation on the aggregated data produces reasonable results.
> />>>/
> />>>/ Right now I'm quite desperate and would appreciate any help.
> />>>/ Thank you
> />>>/ Robert Miller
> />>>/
> />>>/ _______________________________________________
> />>>/ R-sig-mixed-models at r-project.org <https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models> mailing list
> />>>/ https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> />>/
> />>/ --
> />>/ http://mutualism.williams.edu
> />>/
> />>/ _______________________________________________
> />>/ R-sig-mixed-models at r-project.org <https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models> mailing list
> />>/ https://stat.ethz.ch/mailman/listinfo/r-sig-mixed-models
> />>/
> />/
> /
>
>
>
>
>
>
>
>
>
>
> > I like the formality and the fact that this is written down.
> > In a case where a bug is found after release, what would happen: a
> > repeat of the full procedure, a shorter testing phase for the bug fix,
> > or what?
> > -Ian
> >
> > On Wed, Apr 4, 2012 at 9:49 AM, Arni Magnusson <arnima at hafro.is
> > <mailto:arnima at hafro.is>> wrote:
> >
> > I have updated the heading to "Annual Release", and added a couple
> > of explanatory sentences at the top and bottom of the page.
> >
> > Thanks for posting this very useful reference, Johnoel.
> >
> > Arni
> >
> >
> >
> >
> > On Wed, 4 Apr 2012, John Sibert wrote:
> >
> > It looks rather formal.
> >
> > How frequently would there be a new release?
> >
> >
> >
> >
> > On Wed, 3 Ap 2012, Johnoel Ancheta wrote:
> >
> > http://admb-project.org/developers/admb-release-procedure
> >
> >
> > _______________________________________________
> > Developers mailing list
> > Developers at admb-project.org <mailto:Developers at admb-project.org>
> > http://lists.admb-project.org/mailman/listinfo/developers
> >
> >
> >
> > _______________________________________________
> > Developers mailing list
> > Developers at admb-project.org
> > http://lists.admb-project.org/mailman/listinfo/developers
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.admb-project.org/pipermail/developers/attachments/20120404/5aaccbf1/attachment.html>
>
> ------------------------------
>
> _______________________________________________
> Developers mailing list
> Developers at admb-project.org
> http://lists.admb-project.org/mailman/listinfo/developers
>
>
> End of Developers Digest, Vol 38, Issue 4
> *****************************************
--
Matthew Supernaw
Scientific Programmer
National Oceanic and Atmospheric Administration
National Marine Fisheries Service
Sustainable Fisheries Division
St. Petersburg, FL, 33701
Phone 727-551-5606
Fax 727-824-5300
More information about the Developers
mailing list