Subject: Re: What's been done [Re: Voting (again)]

From: Brian E Carpenter <[email-masked]>
Date: Wed, 04 May 2005 15:39:44 +0200

(catching up after a few days in meetings, but it will
still take a while to read everything)

Dave Crocker wrote:
> Brian,
> 
> 
> 
>>> 1.  Apparently you missed the extended, public exchanges about these
>>> issues, over the last 3 years...  
> 
> 
>> Here's a quick list of things that have been done. It's written in
>> somewhat high level terms, but there is substance behind each of these
>> items. It doesn't mean that we're done or that we're complacent, but
>> accusing ourselves of inaction is plain wrong.

First point - I unaccountably forget to mention that we agreed on
and published an IETF Mission Statement (RFC 3935). That was a direct
response to the first "root problem" described in RFC 3774.

> 
> 
> I did not say there had been no activity.  I said that we had done nothing 
> about the issues currently under discussion.  
> 
> So i will ask you and everyone else to consider your list with respect to 
> quality, timeliness and relevance. 
> 
> Most of the actions you cite may affect some of the problematic aspects of 
> timeliness... sometimes. But that is all. 
>   
> Here's an exercise you might want to consider:  Take the list in the Problems 
> RFC and try to reconcile your list against its.  

I do intend to go through that RFC to look for issues that haven't been addressed.
If people would stop submitting drafts for publication and sending technical
email, I might get a chance to do so. But actually, I do prefer to give
priority to the IETF's work product rather than to metawork.

> 
> 
> 
>> Tools team ...
>> Many operational details of document review process ...
> 
> 
> These two are about better administration.  Improving administrative process 
> is fine, but has nothing to do with problems in the "semantics" of the 
> process.  
> 
> It does not affect quality or relevance.  And it does not do much about 
> timeliness, in terms of making working groups go faster, reducing 
> inappropriateintentional blocks by ADs, or any of the things that are the meat 
> of IETF work.  

I disagree. Removing artefacts makes it much easier to look at essentials.
We work now with very convenient hyperlinked agendas, ballots and comments
available on line, etc. On the few occasions when I've had to go back to
more primitive methods, I have found it very distracting and much harder
to quickly and fairly review material. Better tools improve quality
and timeliness. For relevance, see below.
> 
> 
> 
>> and the General Area review team are operational.
> 
> 
> And since it is carefully targeted for the END of the working group effort, 
> how will this improve quality, timeliness and relevance?

Even late reviews can improve quality, and speaking for myself, their availability
makes it much less likely that I will defer a ballot, so they improve
timeliness. I'm not really sure on balance what you mean by relevance.
Relevance to what? Judged by who?
> 
> 
> 
>> But I, Dave and ICAR blew the early review issue so far.)
> 
> 
> Since this was an effort directly targeting quality and timeliness -- and 
> especially since early reviews seem to have succeeded at gaining IETF rough 
> consensus as a Good Thing to do -- do you have an theory about the failure to 
> get this going, or better still, how to get it to succeed?

I think the answer will be baby steps - even with ICAR's modest goals,
it seemed that we couldn't achieve liftoff. The IESG is thinking about
one baby step (pls wait until the notes from our retreat come out).
> 
>  
> 
>> Education team in place to continuously educate leaders and participants
> 
> 
> And you should note that I cited that elsewhere in the thread.  
> 
> Whether the actual content of the training is likely to have any impact on 
> quality, timeliness and relevance is worth assessing.  Yes, it ought to, but 
> how are we going to figure out whether it does?

Very hard. But the EDU team is not complacent either.
> 
> 
> 
>> Regular reviews of RFC Editor and IANA performance in place...
>> New procedures for liaison handling defined...
>> Administrative support unit being created by ISOC...
> 
> 
> And you think these affect core issues of IETF utility to the Internet 
> community?

Yes. The fact that we are slow in getting RFCs out or in making routine
parameter assignments is damaging to the IETF because it makes our users
(implementers and other standards bodies) unhappy. Efficient handling
of liaisons from other bodies is also very important lubrication for the
whole community. And the admin support is tremendously important - partly
to make sure that meetings, mailing lists and document approvals continue to
happen, and partly to free up enough time for *me* to do my job as Chair
and worry about the things you want me to worry about. My personal estimate
is that about 50% of the issues on my current issues list will be
handed off to the IAD when (s)he is up and running.
> 
> You think that these changes will have any effect at all on better IETF 
> participation or producing timely specifications of better quality and 
> relevance?  
> 
> Please explain how.  

Again, relevance is in the eye of the beholder. My quick answer on relevance
is what it's always been - the most important single action we ever take
is chartering a new WG. If we blow that, we risk doing irrelevant work. The
other part of the answer is that it's usually the market that tells us
afterwards what was relevant. But note, the IESG just spent two days
in meetings at the ITU - largely to understand *relevant* requirements from
that community.

So, I think improving operational tools and taking admin burdens away from
the engineering leadership will help us significantly by giving us more
time to think carefully about what work to charter (which directly
impacts participation) and to make the engineering parts of our process
work better.

However, we depend tremendously on skilful and dedicated WG chairs and
participants. Without them, no process and no administration will
succeed.
> 
> You might consider using the Problems RFC as a reference guide for directing 
> such an assessment.
> 
> 
> 
> 
>> Proposals for upgrading/streamlining standards track in discussion (i.e.
>> newtrk and specifically the ISD proposal, but there's certainly more to do
>> in newtrk)
> 
> 
> Another derailed activity.  Another activity that has nothing to do with 
> quality, timeliness or relevance.
> 
> Go ahead, explain how it does.  

John Klensin answered this one.
> 
> Explain how current document labeling practises hurt the IETF's utility to the 
> Internet community.

Actually, RFC 3774 dinged the 3 stage standards process, not me.
> 
> Explain how the utility of the IETF to the community will be improved by our 
> fixing this.

Indeed, this something the IESG also discussed in its retreat; and you've
seem John's answer.
> 
> Brian, we need to distinguish activity from progress. 

Now there's a truism.
> 
> We need to look for the issues that are at the core of the IETF's 
> problems.  
> 
> As I said, in the last 3+ years, we have mostly chosen not to do 
> that.

On the contrary, we're working through them. But personally I'm not
panicking, nor am I complacent.

    Brian

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: "JFC (Jefsey) Morfin" <[email-masked]>
Date: Wed, 04 May 2005 16:39:34 +0200

On 15:39 04/05/2005, Brian E Carpenter said:
>My quick answer on relevance is what it's always been - the most important 
>single action we ever take is chartering a new WG.

Yes. Also to make sure the WG reviewed it and consensually understood it 
the same way. And that the IESG understood the possible amendment of the WG 
from the experience of its members. Before starting working.
jfc

---

From: Dave Crocker <[email-masked]>
Date: Wed, 4 May 2005 13:12:02 -0700

Brian,

>  First point - I unaccountably forget to mention that we agreed on and
>  published an IETF Mission Statement (RFC 3935). That was a direct response
>  to the first "root problem" described in RFC 3774.

Do you believe that that document will meaningfully contribute to the IETF's 
producing documents that are more timely and relevent and have better quality?

If so, it would be enlightening to hear how and in what timeframe.

>   If people would stop submitting drafts for publication and
>  sending technical email, I might get a chance to do so. But actually, I do
>  prefer to give priority to the IETF's work product rather than to metawork.

I suspect that it is not obvious to folks just how significant your comment 
is.  It translates into:  

   "Senior IETF management cannot devote much energy to fixing its strategic 
timeliness, relevance and quality issues, because it is too busy attending to 
the daily tactics of process that we agree are deficient in producing timely, 
relevant documents of high quality."

And, yes, I know you don't mean that, but in fact that is the reality your 
statement reflects.

FWIW, it is a classic dilemma in organizations' efforts to remedy strategic 
problems, since most organizations feel the press of daily deliverables.

> >  inappropriateintentional blocks by ADs, or any of the things that are
> >  the meat of IETF work.
>

Brian, it has been about 3 years since the IETF acknowledged it had a crisis.  
Are we focusing any better on the essentials now?  

I think we are focusing less. When do you project that this will change?

Fixing mechanical details definitely makes life easier.  No doubt about it.

But it is folly to think that any of that is really going to get us to focus 
on essentials more, at some indefinite time in the future.

Mechanical details do not get working groups to focus better, consult experts 
better, or produce better documents more quickly.  Really, they don't.

Mechanical details do not get ADs to take a more facilitative stance. Really, 
they don't.

>  Even late reviews can improve quality, and speaking for myself, their
>  availability makes it much less likely that I will defer a ballot, so they
>  improve timeliness. 

The key word you used is "can".  Unfortunately, an one-time (or rare) 
demonstration of an existence proof for utility is not enough.  These 
processes are expensive.  If they do not have regular, substantial benefit, we 
should be spending our process dollars elsewhere.

If we turn out better documents as a result of normal processes, we will not 
need late-stage reviews because the benefits will have been realized sooner 
(and probably better and cheaper.)

>I'm not really sure on balance what you mean by
>  relevance. Relevance to what? Judged by who?

The IETF turns out a lot of documents.  We do not pay much attention to their 
long-term utility.  There is a general sense that the percentage of our output 
that is useful has gone down dramatically.  

That means our work is far less relevant.

An excellent spec that is too late for the market is not relevant.  A spec 
that provides for a mechanism that is too cumbersome to administer is not 
relevant.

And so on.

> > >  But I, Dave and ICAR blew the early review issue so far.)
....
>  I think the answer will be baby steps - even with ICAR's modest goals, it
>  seemed that we couldn't achieve liftoff. The IESG is thinking about one
>  baby step (pls wait until the notes from our retreat come out).

It really is facinating just how quickly you seem willing to give up on 
efforts like this.  

As I've noted in a separate posting earlier, we spend the better part of a 
decade nursing along failed specification efforts, but we seem to give these 
sorts of strategic change efforts about 3 months to succeed.  

Ok, maybe 6...

> > >  Regular reviews of RFC Editor and IANA performance in place... New
> > >  procedures for liaison handling defined... Administrative support
> > >  unit being created by ISOC...
>  (implementers and other standards bodies) unhappy. Efficient handling of
>  liaisons from other bodies is also very important lubrication for the
>  to make sure that meetings, mailing lists and document approvals continue
>  to happen, and partly to free up enough time for *me* to do my job as
>  Chair and worry about the things you want me to worry about. My personal
>  estimate is that about 50% of the issues on my current issues list will be

Brian, I have no doubt that these are all perfectly reasonable to make better. 

But do you have any basis for saying that problems with ANY of these issues 
has created a pattern of delay in producing specifications or produced worse 
specifications or produced specifications that were less relevant to the 
community?

What I am suggesting, here, is that you are focusing on a myriad of details 
that are, in fact, of tactical importance to the survival of the IETF, not of 
strategic importance.

Tactics are of course significant.  But not in the absence of serious efforts 
to fix deep, strategic problems.

> >  Brian, we need to distinguish activity from progress.

and since we all know that, it's probably worth thinking about my choosing to 
express it at this juncture.

  d/
  ---
  Dave Crocker
  Brandenburg InternetWorking
  +1.408.246.8253
  dcrocker  a t ...
  WE'VE MOVED to:  www.bbiw.net