From owner-ips@ece.cmu.edu Thu Feb 1 05:34:14 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id FAA23191
for ; Thu, 1 Feb 2001 05:34:14 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f118Pba09913
for ips-outgoing; Thu, 1 Feb 2001 03:25:37 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f118PCZ09876
for ; Thu, 1 Feb 2001 03:25:13 -0500 (EST)
Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23])
by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id JAA46336
for ; Thu, 1 Feb 2001 09:25:05 +0100
From: julian_satran@il.ibm.com
Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237])
by d12relay02.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id JAA73160
for ; Thu, 1 Feb 2001 09:25:05 +0100
Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569E6.002E391D ; Thu, 1 Feb 2001 09:24:52 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: ips@ece.cmu.edu
Message-ID:
Date: Thu, 1 Feb 2001 10:20:24 +0200
Subject: Re: iSCSI : Digest Error recovery causes data corruption
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Pierre,
Please don't put salt on my wounds... (and explicitly for the reasons you
mentioned) I suggested that back in San Diego. There where 2 reason for
abandoning it:
1- it can be practically done without TCP modification if CRCs are put in
the IPsec Authentication Header but the IPsec people won't have anything
weaker that any of the cryptographic digests and without IPsec we are out
of options (no change to TCP!)
2-more important Storage Proxies would like to have their separate data
digest (or equivalent)
Regards,
Julo
Pierre Labat on 31/01/2001 20:19:17
Please respond to Pierre Labat
To: ips@ece.cmu.edu
cc:
Subject: iSCSI : Digest Error recovery causes data corruption
Hello,
I would invite you to revisit the Mike Krause mail about
"iSCSI Data Integrity - Digests".
If Alternative 1 or 2 is adopted, when a digest error (CRC)
occurs, the segment could be discarded, and TCP will
retransmit. For software implementation with an hardware
assist to calculate the CRC, (same kind of the one that exists
now to calculate the checksum now), if a CRC error is detected
it could be assimilated as a checksum error.
Hence there is no need to overload the iSCSI protocol
to deal with digest errors, because the iSCSI layer could not
see them.
Why to do complicate when simple an efficient can be achieved?
Regards,
Pierre
From owner-ips@ece.cmu.edu Thu Feb 1 06:12:08 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id GAA23468
for ; Thu, 1 Feb 2001 06:12:08 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f1192cR11230
for ips-outgoing; Thu, 1 Feb 2001 04:02:38 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f1191hZ11191
for ; Thu, 1 Feb 2001 04:01:43 -0500 (EST)
Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22])
by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id KAA29368
for ; Thu, 1 Feb 2001 10:01:31 +0100
From: julian_satran@il.ibm.com
Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237])
by d12relay01.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id KAA23606
for ; Thu, 1 Feb 2001 10:01:31 +0100
Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569E6.00319173 ; Thu, 1 Feb 2001 10:01:24 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: ips@ece.cmu.edu
Message-ID:
Date: Thu, 1 Feb 2001 10:38:44 +0200
Subject: Re: iSCSI: Text Commands
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Josh,
Thanks. I will add a statement to this effect.
Julo
Joshua Tseng on 31/01/2001 20:41:11
Please respond to Joshua Tseng
To: ips@ece.cmu.edu
cc:
Subject: iSCSI: Text Commands
Julo,
I think a statement needs to be added that the iSCSI initiator MUST support
all of the text commands listed in Appendix C. Currently, it is unclear
whether
a target can respond with key:value pairs that were not solicited in the
original
text command. The spec should explicitly say the target can do so, and
require the initiator to recognize all text keys. For example, if the
target wants
to specify a FirstBurst of less than 65535 (BTW, why is the the default
value
for FirstBurst so high???), it should be able to specify its own value for
FirstBurst,
even if the "FirstBurst:" text key was not listed in the original text
command.
Josh
From owner-ips@ece.cmu.edu Thu Feb 1 09:07:49 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id JAA27440
for ; Thu, 1 Feb 2001 09:07:47 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11BtgO27062
for ips-outgoing; Thu, 1 Feb 2001 06:55:42 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11BtaZ27057
for ; Thu, 1 Feb 2001 06:55:36 -0500 (EST)
Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23])
by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id MAA118150;
Thu, 1 Feb 2001 12:55:27 +0100
From: julian_satran@il.ibm.com
Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237])
by d12relay02.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id MAA225826;
Thu, 1 Feb 2001 12:55:27 +0100
Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569E6.00417B8D ; Thu, 1 Feb 2001 12:55:14 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: ips@ece.cmu.edu
cc: ralphoweber@compuserve.com
Message-ID:
Date: Thu, 1 Feb 2001 12:42:26 +0200
Subject: Re: iSCSI : Holes in StatSN
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Santosh,
Overlaps and out of order delivery and gaps are not forbidden by SAM . I
think we have to go to T10 for that I can't see a good reason to do it. We
have a good solution without asking for it . I can see large and important
future application that will relay on overlaps and/or gaps and I am not
going to foolishly do something to disable or harm their efficient
implementation. I think that T10s philosophy of keeping the target master
of the transfer and not limiting it in any way is too valuable to ignore.
IMHO your request violates our charter without any good reason to support
it.
Julo
Santosh Rao on 01/02/2001 03:53:13
Please respond to Santosh Rao
To: ips@ece.cmu.edu
cc:
Subject: Re: iSCSI : Holes in StatSN
julian_satran@il.ibm.com wrote:
> With a data sequence we may want to use a similar mechanism to ask for a
> missed data block as soon as we see one of its successors or the status.
Julian,
The missing data PDU is missing due to either a header format error, header
digest error or data digest error. [in all other cases, TCP ensures
reliable
delivery].
In 2 of the above 3 cases, [header format error & header digest error] the
initiator CANNOT do a safe interpretation of the PDU header. Without
interpreting the PDU header, the initiator does NOT get the Initiator Task
Tag. Any request to re-send a particular data PDU MUST be qualified by :
I.T.T + missing_DataSN [+ T.T.T + CmdSN, optionally].
Since I.T.T. cannot be reliably determined in 2 of the 3 cases, such a
re-send request cannot be reliably achieved.
The alternate proposal that was made should be considered in its place,
which
was to :
- dis-allow overlapped data xfer's
- initiators do a count check
- a command level retry is performed at the iSCSI layer on detecting an
underrun [due to a missing PDU].
On several ocassions, requests from different people have been made on this
list to dis-allow overlapped data xfers. Can a WG consensus be sought on
this
issue to see if the benefits of allowing overlapped data xfer's offset its
complexities and justify its support ?
Regards,
Santosh
- santoshr.vcf
From owner-ips@ece.cmu.edu Thu Feb 1 13:50:17 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA07202
for ; Thu, 1 Feb 2001 13:50:14 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11GEpD09969
for ips-outgoing; Thu, 1 Feb 2001 11:14:51 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from palrel3.hp.com (palrel3.hp.com [156.153.255.226])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11GEQZ09952
for ; Thu, 1 Feb 2001 11:14:28 -0500 (EST)
Received: from hpindlm.cup.hp.com (hpindlm.cup.hp.com [15.13.95.89])
by palrel3.hp.com (Postfix) with ESMTP
id D4600717; Thu, 1 Feb 2001 08:14:06 -0800 (PST)
Received: from mk731913.cup.hp.com (mk731912.cup.hp.com [15.8.80.111])
by hpindlm.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id IAA02266;
Thu, 1 Feb 2001 08:16:44 -0800 (PST)
Message-Id: <5.0.0.25.2.20010201073752.02800e88@hpindlm.cup.hp.com>
X-Sender: krause@hpindlm.cup.hp.com
X-Mailer: QUALCOMM Windows Eudora Version 5.0
Date: Thu, 01 Feb 2001 08:01:58 -0800
To: Sean Quinlan
From: Michael Krause
Subject: Re: iSCSI Data Integrity - Digests
Cc: ips@ece.cmu.edu
In-Reply-To: <3A763A5B.B11ADDA6@bell-labs.com>
References: <5.0.0.25.2.20010126165115.00a73a80@hpindlm.cup.hp.com>
<5.0.0.25.2.20010129124010.023ce6d0@hpindlm.cup.hp.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
At 10:51 PM 1/29/2001 -0500, Sean Quinlan wrote:
>I would suggest that if iSCSI requires that a PDU does not span a
>TCP segment then iSCSI is indeed changing TCP. A valid implementations
>of TCP has considerable freedom in deciding how to pack data into segments
>and this is not under the control of the application layer. Such a
>requirement in iSCSI would mean it is not feasible to implement iSCSI on
>many existing TCP stacks.
The continual assertion that TCP implementations are sacrosanct and thus
not cannot be modified to support iSCSI combined with a tendency to make
everything an option should someone object quite frankly leads one to
question whether iSCSI will suffer the same 10-year growing pains /
interoperability problems FC suffered and thus seriously raises a question
of its viability within the industry.
Yes, iSCSI could be implemented using SCTP which is, from an architecture
point of view, more suitable in many ways than TCP. However, SCTP isn't
really deployed today, is not targeted by those few that have deployed it
for a high-volume market, and thus unclear whether it can generate
sufficient interest to make it a viable transport from a business
perspective. I don't want to debate SCTP vs. TCP - there has been enough
of it already on this reflector and the only point was while it is
interesting, the business issues are a major problem that is not going to
be solved in the desired deployment time frames.
Summary: Either iSCSI starts getting realistic in terms of what is subject
to change without violating / major modifications to existing protocols
(not implementations) and starts reducing the number of options so
interoperability has a chance of success or it will suffer the fate of FC's
early years.
>If you change the requirements of for a valid TCP implementation I believe
>you have effectively changed the protocol.
The protocol is defined as a set of wire formats and semantics. The
proposal did not violate any aspect of the RFC which does allow TCP a great
deal of freedom in terms of how it packages segments.
>I agree with Douglas Otis that such a change may not be well received.
>
>What would you suggest for situations such a iSCSI over a protocol such as
>TLS(SSL).
>TLS uses an internal framing mechanism that is not under the applications
>control and is also not related to the underlying TCP segments.
Put some intelligence into the solution's implementation and segment
accordingly.
>It would seem to me a little unwise to mandate end-to-end data integrity
>within iSCSI
>especially using a CRC algorithm. Rather a lot of data goes over protocols
>such as HTTP, FTP, NFS and CIFS without any additional end-to-end data
>integrity
>over what is provided by TCP and link level integrity - I think it is a little
>strong to say
> strong end-to-end data integrity is not an option as customers will
> not adopt solutions without such guarantees.
The original proposal stated what the difference here is: these protocols
target buffers and associated information that is not transmitted over the
wire thus reducing the probability of a DMA targeting an incorrect location
in memory. iSCSI communicates addressing information either direct VAs or
a handle for the look-up. In either case, there is a greater probability
of a DMA targeting the wrong location as a result. FC protects their data
with a 32-bit CRC which provides customers with a lot greater confidence in
the data integrity than what they get from TCP's checksum.
Also, as noted by numerous people there is a measured evidence of silent
data corruption getting past TCP's checksum today in the Internet. If
people really believe iSCSI will flow across the Internet, then silent data
corruption must be dealt with in a much stronger manner than what is being
proposed and having it as an option is unacceptable for most vendors and
customers.
>If TLS or IPsec are used in conjunction with iSCSI, then strong
>end-to-end integrity is already provided and duplicating this in iSCSI seem
>rather a waste. It may be the case that some people are not satisfied with
>the integrity provided by TCP and yet do not want to use TLS or IPsec to
>ensure
>integrity/authenticity, but I suspect that such people are not the
>majority and
>thus mandating a data integrity mechanism within iSCSI seems like a mistake -
>perhaps as an option or perhaps leave it to other layers in the protocol
>stack.
One cannot mandate IPSec / TLS usage within the standard nor can one rely
upon it for data integrity. To assume that the majority of people want it
is incorrect as well since this is a function of the usage models being
deployed. Many people interested in iSCSI are not interested per se in
block storage over the actual Internet but block storage over IP-based
networks - there is a very big difference here and the requirements for
security as a result.
>If data integrity is provided by iSCSI, CRC algorithms are not particularly
>ideal from a software implementation point of view - surely there
>are better alternatives than CRC that are easy to implement in hardware and
>can take advantage of 32bit wide data ops.
CRCs are well understood. While software is an interesting implementation
and certainly one that I do not want to see harmed, many vendors are
developing hardware-based solutions (keeps the CPU consumption to be the
same as FC which is important for customers and vendors). The proposal put
forth also provided a mechanism for some intelligence to be added to
non-iSCSI hardware implementations similar to the way one does checksum
off-load today allowing software to not necessarily be burdened with
unnecessary overheads if so desired.
Mike
From owner-ips@ece.cmu.edu Thu Feb 1 16:04:39 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA11528
for ; Thu, 1 Feb 2001 16:04:37 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11J1J217646
for ips-outgoing; Thu, 1 Feb 2001 14:01:19 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from palrel3.hp.com (palrel3.hp.com [156.153.255.226])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11IxoZ17549
for ; Thu, 1 Feb 2001 13:59:50 -0500 (EST)
Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203])
by palrel3.hp.com (Postfix) with ESMTP id 5931973D
for ; Thu, 1 Feb 2001 10:59:49 -0800 (PST)
Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197])
by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id KAA00779
for ; Thu, 1 Feb 2001 10:59:44 -0800 (PST)
Message-ID: <3A79B2C6.2FD26DC7@cup.hp.com>
Date: Thu, 01 Feb 2001 11:02:30 -0800
From: Santosh Rao
Organization: Hewlett Packard, Cupertino.
X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778)
X-Accept-Language: en
MIME-Version: 1.0
Cc: ips@ece.cmu.edu
Subject: Re: iSCSI : Holes in StatSN
References:
Content-Type: multipart/mixed;
boundary="------------0E5B87DD0869E3B98E81E4DE"
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
This is a multi-part message in MIME format.
--------------0E5B87DD0869E3B98E81E4DE
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Julian,
I did'nt hear back on my issues related to attempting per-PDU recovery, namely
:
a) This goes back to attempting to recover a portion of the I/O as opposed to
command level recovery. I believe the WG consensus at Orlando was to use
command level recovery (?)
b) A request to re-send a Data PDU within an I.T.T must be qualified by (I.T.T
+ missing_DataSN).
The I.T.T & DataSN CANNOT be reliably extracted from the PDU in the cases of a
header format error or header digest error. IOW, the per-PDU scheme does not
solve all the cases.
Regarding overlapped data xfers, FCP prohibited it and seems to have no
problems with this. Also, this feature is un-used in most cases. I don't
believe restricting iSCSI to support a sub-set of the optional features of
SAM-2 violates either T10 philosophy or SAM-2. OTOH, specifying guidelines
that mandate which features shall be used and un-used (along the lines of
FC-PLDA and FC-FLA documents) is the only reliable recipe for
interoperability.
Regards,
Santosh
julian_satran@il.ibm.com wrote:
> Santosh,
>
> Overlaps and out of order delivery and gaps are not forbidden by SAM . I
> think we have to go to T10 for that I can't see a good reason to do it. We
> have a good solution without asking for it . I can see large and important
> future application that will relay on overlaps and/or gaps and I am not
> going to foolishly do something to disable or harm their efficient
> implementation. I think that T10s philosophy of keeping the target master
> of the transfer and not limiting it in any way is too valuable to ignore.
>
> IMHO your request violates our charter without any good reason to support
> it.
>
> Julo
>
> Santosh Rao on 01/02/2001 03:53:13
>
> Please respond to Santosh Rao
>
> To: ips@ece.cmu.edu
> cc:
> Subject: Re: iSCSI : Holes in StatSN
>
> julian_satran@il.ibm.com wrote:
>
> > With a data sequence we may want to use a similar mechanism to ask for a
> > missed data block as soon as we see one of its successors or the status.
>
> Julian,
>
> The missing data PDU is missing due to either a header format error, header
> digest error or data digest error. [in all other cases, TCP ensures
> reliable
> delivery].
>
> In 2 of the above 3 cases, [header format error & header digest error] the
> initiator CANNOT do a safe interpretation of the PDU header. Without
> interpreting the PDU header, the initiator does NOT get the Initiator Task
> Tag. Any request to re-send a particular data PDU MUST be qualified by :
> I.T.T + missing_DataSN [+ T.T.T + CmdSN, optionally].
>
> Since I.T.T. cannot be reliably determined in 2 of the 3 cases, such a
> re-send request cannot be reliably achieved.
>
> The alternate proposal that was made should be considered in its place,
> which
> was to :
> - dis-allow overlapped data xfer's
> - initiators do a count check
> - a command level retry is performed at the iSCSI layer on detecting an
> underrun [due to a missing PDU].
>
> On several ocassions, requests from different people have been made on this
> list to dis-allow overlapped data xfers. Can a WG consensus be sought on
> this
> issue to see if the benefits of allowing overlapped data xfer's offset its
> complexities and justify its support ?
>
> Regards,
> Santosh
>
> - santoshr.vcf
--------------0E5B87DD0869E3B98E81E4DE
Content-Type: text/x-vcard; charset=us-ascii;
name="santoshr.vcf"
Content-Description: Card for Santosh Rao
Content-Disposition: attachment;
filename="santoshr.vcf"
Content-Transfer-Encoding: 7bit
begin:vcard
n:Rao;Santosh
tel;work:408-447-3751
x-mozilla-html:FALSE
org:Hewlett Packard, Cupertino.;SISL
adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA.
version:2.1
email;internet:santoshr@cup.hp.com
title:Software Design Engineer
x-mozilla-cpt:;21088
fn:Santosh Rao
end:vcard
--------------0E5B87DD0869E3B98E81E4DE--
From owner-ips@ece.cmu.edu Thu Feb 1 16:05:06 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA11539
for ; Thu, 1 Feb 2001 16:05:04 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11Jnpo20104
for ips-outgoing; Thu, 1 Feb 2001 14:49:51 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11JnQZ20079
for ; Thu, 1 Feb 2001 14:49:26 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f11Ktu065333;
Thu, 1 Feb 2001 12:55:56 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: "Michael Krause" , "Sean Quinlan"
Cc:
Subject: RE: iSCSI Data Integrity - Digests
Date: Thu, 1 Feb 2001 11:44:17 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
In-Reply-To: <5.0.0.25.2.20010201073752.02800e88@hpindlm.cup.hp.com>
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Mike,
I agree with most of your points with the exception of TCP being a wire
protocol. Use the SCTP structures and apply them to TCP where the SCTP port
locations become a pseudo frame word length and then allow padding to a
predefined interval and you then have the required framing and all the
benefits of SCTP while not harming TCP at the API. As we get mired down in
amorphic iSCSI, a transport specification as complete as SCTP would
immediately provide productive solutions and a clean migration to that
uncharted land where SCTP dominates.
Doug
> At 10:51 PM 1/29/2001 -0500, Sean Quinlan wrote:
> >I would suggest that if iSCSI requires that a PDU does not span a
> >TCP segment then iSCSI is indeed changing TCP. A valid implementations
> >of TCP has considerable freedom in deciding how to pack data
> into segments
> >and this is not under the control of the application layer. Such a
> >requirement in iSCSI would mean it is not feasible to implement iSCSI on
> >many existing TCP stacks.
>
> The continual assertion that TCP implementations are sacrosanct and thus
> not cannot be modified to support iSCSI combined with a tendency to make
> everything an option should someone object quite frankly leads one to
> question whether iSCSI will suffer the same 10-year growing pains /
> interoperability problems FC suffered and thus seriously raises a
> question
> of its viability within the industry.
>
> Yes, iSCSI could be implemented using SCTP which is, from an architecture
> point of view, more suitable in many ways than TCP. However, SCTP isn't
> really deployed today, is not targeted by those few that have deployed it
> for a high-volume market, and thus unclear whether it can generate
> sufficient interest to make it a viable transport from a business
> perspective. I don't want to debate SCTP vs. TCP - there has been enough
> of it already on this reflector and the only point was while it is
> interesting, the business issues are a major problem that is not going to
> be solved in the desired deployment time frames.
>
> Summary: Either iSCSI starts getting realistic in terms of what
> is subject
> to change without violating / major modifications to existing protocols
> (not implementations) and starts reducing the number of options so
> interoperability has a chance of success or it will suffer the
> fate of FC's
> early years.
>
> >If you change the requirements of for a valid TCP implementation
> I believe
> >you have effectively changed the protocol.
>
> The protocol is defined as a set of wire formats and semantics. The
> proposal did not violate any aspect of the RFC which does allow
> TCP a great
> deal of freedom in terms of how it packages segments.
>
> >I agree with Douglas Otis that such a change may not be well received.
> >
> >What would you suggest for situations such a iSCSI over a
> protocol such as
> >TLS(SSL).
> >TLS uses an internal framing mechanism that is not under the applications
> >control and is also not related to the underlying TCP segments.
>
> Put some intelligence into the solution's implementation and segment
> accordingly.
>
>
> >It would seem to me a little unwise to mandate end-to-end data integrity
> >within iSCSI
> >especially using a CRC algorithm. Rather a lot of data goes
> over protocols
> >such as HTTP, FTP, NFS and CIFS without any additional end-to-end data
> >integrity
> >over what is provided by TCP and link level integrity - I think
> it is a little
> >strong to say
> > strong end-to-end data integrity is not an option as
> customers will
> > not adopt solutions without such guarantees.
>
> The original proposal stated what the difference here is: these protocols
> target buffers and associated information that is not transmitted
> over the
> wire thus reducing the probability of a DMA targeting an
> incorrect location
> in memory. iSCSI communicates addressing information either
> direct VAs or
> a handle for the look-up. In either case, there is a greater probability
> of a DMA targeting the wrong location as a result. FC protects
> their data
> with a 32-bit CRC which provides customers with a lot greater
> confidence in
> the data integrity than what they get from TCP's checksum.
>
> Also, as noted by numerous people there is a measured evidence of silent
> data corruption getting past TCP's checksum today in the Internet. If
> people really believe iSCSI will flow across the Internet, then
> silent data
> corruption must be dealt with in a much stronger manner than what
> is being
> proposed and having it as an option is unacceptable for most vendors and
> customers.
>
> >If TLS or IPsec are used in conjunction with iSCSI, then strong
> >end-to-end integrity is already provided and duplicating this in
> iSCSI seem
> >rather a waste. It may be the case that some people are not
> satisfied with
> >the integrity provided by TCP and yet do not want to use TLS or IPsec to
> >ensure
> >integrity/authenticity, but I suspect that such people are not the
> >majority and
> >thus mandating a data integrity mechanism within iSCSI seems
> like a mistake -
> >perhaps as an option or perhaps leave it to other layers in the protocol
> >stack.
>
> One cannot mandate IPSec / TLS usage within the standard nor can
> one rely
> upon it for data integrity. To assume that the majority of people want it
> is incorrect as well since this is a function of the usage models being
> deployed. Many people interested in iSCSI are not interested per se in
> block storage over the actual Internet but block storage over IP-based
> networks - there is a very big difference here and the requirements for
> security as a result.
>
>
> >If data integrity is provided by iSCSI, CRC algorithms are not
> particularly
> >ideal from a software implementation point of view - surely there
> >are better alternatives than CRC that are easy to implement in
> hardware and
> >can take advantage of 32bit wide data ops.
>
> CRCs are well understood. While software is an interesting
> implementation
> and certainly one that I do not want to see harmed, many vendors are
> developing hardware-based solutions (keeps the CPU consumption to be the
> same as FC which is important for customers and vendors). The
> proposal put
> forth also provided a mechanism for some intelligence to be added to
> non-iSCSI hardware implementations similar to the way one does checksum
> off-load today allowing software to not necessarily be burdened with
> unnecessary overheads if so desired.
>
> Mike
>
>
From owner-ips@ece.cmu.edu Thu Feb 1 16:05:19 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA11550
for ; Thu, 1 Feb 2001 16:05:17 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11Io6k17057
for ips-outgoing; Thu, 1 Feb 2001 13:50:06 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11In1Z17002
for ; Thu, 1 Feb 2001 13:49:01 -0500 (EST)
Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203])
by palrel1.hp.com (Postfix) with ESMTP id 4C1CE485
for ; Thu, 1 Feb 2001 10:48:56 -0800 (PST)
Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197])
by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id KAA29353
for ; Thu, 1 Feb 2001 10:48:49 -0800 (PST)
Message-ID: <3A79B036.462E8F12@cup.hp.com>
Date: Thu, 01 Feb 2001 10:51:35 -0800
From: Santosh Rao
Organization: Hewlett Packard, Cupertino.
X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778)
X-Accept-Language: en
MIME-Version: 1.0
To: IPS Reflector
Subject: Re: iSCSI : Holes in StatSN
References:
Content-Type: multipart/mixed;
boundary="------------59DA8355A6A63836EBA8C8EC"
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
This is a multi-part message in MIME format.
--------------59DA8355A6A63836EBA8C8EC
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
julian_satran@il.ibm.com wrote:
> The sack will specify a range to be filled. The initiator will revert to
> bulk as soon as it has no holes.
Julian,
Once a StatSN hole is created, it is never filled. [since subsequent "retry"
of the command results in a different StatSN being used.]. However, there is
currently no association available to determine when it is safe to
acknowledge that hole [which happens when the initiator switches back to bulk
ACK scheme.]
Given the above, the initiator needs to maintain a list of Initiator Task
Tags for which StatSN holes were encountered (i.e. Status PDU was not
received). It then needs to dequeue elements from this list when the "retry"
command completes or the command is aborted. Once all elements are off this
list, it can revert to the bulk StatSN ACK mechanism.
> A target can also resend after a timeout or when seen the same StatSN on a
> NOP.
Not sure what is implied here. Surely, we are not attempting a re-transmit
functionality akin to TCP re-transmit at the iSCSI layer for Status PDUs (?)
> The initiator will query after
> a long silence with a NOP (not longer than the SCSI timeout -:))
This is intriguing. Are you suggesting that on an I/O timeout, the initiator
send a NOP-OUT to request a re-transmit of the Status PDU ? How does the
initiator know one of the Data PDUs also did not time out ?
(An I/O timeout also implies that the O.S. expects a response back without
any further time being spent on the I/O. I/O timeouts cause the initiator to
abort the I/O and error the I/O up the stack. )
Regards,
Santosh
> Santosh Rao on 01/02/2001 02:57:25
>
> Please respond to Santosh Rao
>
> To:
> cc: ips@ece.cmu.edu
> Subject: Re: iSCSI : Holes in StatSN
>
> Julian,
>
> Once the selective StatSN ACK mechanism kicks in, how is the initiator to
> revert to the bulk StatSN ACK ? (i.e. when/how does an initiator realize
> that
> the hole is filled ?) Or, does it only use selective StatSN ACK from there
> on
> ?
>
> The difficulty lies in the fact that a hole created will never be filled
> since "retry" will result in target sending back a subsequent Status PDU
> with
> a different StatSN. However, the initiator does not know when to safely
> claim
> that the hole is filled (by sending a bulk StatSN ACK), since there is no
> way
> to detect this.
>
> Regards,
> Santosh
>
--------------59DA8355A6A63836EBA8C8EC
Content-Type: text/x-vcard; charset=us-ascii;
name="santoshr.vcf"
Content-Description: Card for Santosh Rao
Content-Disposition: attachment;
filename="santoshr.vcf"
Content-Transfer-Encoding: 7bit
begin:vcard
n:Rao;Santosh
tel;work:408-447-3751
x-mozilla-html:FALSE
org:Hewlett Packard, Cupertino.;SISL
adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA.
version:2.1
email;internet:santoshr@cup.hp.com
title:Software Design Engineer
x-mozilla-cpt:;21088
fn:Santosh Rao
end:vcard
--------------59DA8355A6A63836EBA8C8EC--
From owner-ips@ece.cmu.edu Thu Feb 1 16:05:59 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA11562
for ; Thu, 1 Feb 2001 16:05:57 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11JS5d18949
for ips-outgoing; Thu, 1 Feb 2001 14:28:05 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from smtp5.mail.yahoo.com (smtp5.mail.yahoo.com [128.11.69.102])
by ece.cmu.edu (8.11.0/8.10.2) with SMTP id f11JRSZ18917
for ; Thu, 1 Feb 2001 14:27:28 -0500 (EST)
Received: from unknown (HELO somesh) (206.112.111.132)
by smtp.mail.vip.suc.yahoo.com with SMTP; 1 Feb 2001 19:27:25 -0000
X-Apparently-From:
Reply-To:
From: "Somesh Gupta"
To: "Matt Wakeley" ,
Subject: RE: iSCSI : Digest Error recovery causes data corruption
Date: Thu, 1 Feb 2001 11:24:26 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700
In-Reply-To: <3A7860C3.E0764D73@agilent.com>
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
One thing I would like to point out is that in the presence of any
kind of middle boxes (isn't the CRC supposed to protect against
that primarily?), the iSCSI data CRC has been proposed to provide
an end-to-end integrity, whereas a TCP connection actually might
exist from a middle box to the receiving end-system.
In this case, retransmission at the TCP layer may continue to
send corrupted data if the corruption happened somewhere prior
to the sending point on the middle-box.
> -----Original Message-----
> From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
> Matt Wakeley
> Sent: Wednesday, January 31, 2001 11:00 AM
> To: ips@ece.cmu.edu
> Subject: Re: iSCSI : Digest Error recovery causes data corruption
>
>
> At first glance, the idea is nice.
>
> However, options 1 and 2 require changes to the implementation
> and API of TCP
> to provide the framing. As you (should) know, we are not allowed
> to change
> TCP. Another problem is that your proposal to drop segments that
> have digest
> errors violates the network layering. TCP will deliver the PDUs
> to iSCSI.
> iSCSI is the layer that checks the CRC. When TCP delivers the
> data to iSCSI,
> it has already acknowledged the segment, so it can't be discarded and
> retransmitted. There is no way for iSCSI to tell TCP to accept or drop a
> segment.
>
> [all of the above assumes existing TCP definition/implementation.
> Anything is
> possible if one is allowed to change things]
>
> -Matt
>
> Pierre Labat wrote:
> >
> > Hello,
> >
> > I would invite you to revisit the Mike Krause mail about
> > "iSCSI Data Integrity - Digests".
> >
> > If Alternative 1 or 2 is adopted, when a digest error (CRC)
> > occurs, the segment could be discarded, and TCP will
> > retransmit. For software implementation with an hardware
> > assist to calculate the CRC, (same kind of the one that exists
> > now to calculate the checksum now), if a CRC error is detected
> > it could be assimilated as a checksum error.
> > Hence there is no need to overload the iSCSI protocol
> > to deal with digest errors, because the iSCSI layer could not
> > see them.
> >
> > Why to do complicate when simple an efficient can be achieved?
> >
> > Regards,
> >
> > Pierre
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
From owner-ips@ece.cmu.edu Thu Feb 1 16:50:11 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA12287
for ; Thu, 1 Feb 2001 16:50:10 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11JSpb18980
for ips-outgoing; Thu, 1 Feb 2001 14:28:51 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11JRBZ18905
for ; Thu, 1 Feb 2001 14:27:11 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f11KbX065314;
Thu, 1 Feb 2001 12:37:33 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: ,
Cc:
Subject: RE: iSCSI : Holes in StatSN
Date: Thu, 1 Feb 2001 11:25:54 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
In-Reply-To:
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Julian,
While I agree data delivery directed by the target with significant freedom
is the convention used by SCSI, Santosh also speaks to a much larger issue.
The present iSCSI transport is not reliable and repairing this weakness
using the SCSI layer is a poor technique. Abandon a mixture of SCSI and
iSCSI and make iSCSI a self sustaining independent layer. View iSCSI as a
general transport that allows handling of digest errors, validation,
multiple connections, and sequential delivery then its header may look
something like this:
+---------------+---------------+---------------+---------------+
0| Type | Reserved | Word Length |
+---------------+---------------+---------------+---------------+
4| Validation Tag |
+---------------+---------------+---------------+---------------+
8| Checksum (Adler32) |
+---------------+---------------+---------------+---------------+
12| ->ClientSN or <-ServerSN |
+---------------+---------------+---------------+---------------+
16| ->ExpServerSN or <-ExpClientSN |
+---------------+---------------+---------------+---------------+
20| ->MaxServerSN or <-MaxClientSN |
+---------------+---------------+---------------+---------------+
+---------------+---------------+---------------+---------------+
24| (Frame Check) |
+---------------+---------------+---------------+---------------+
28| SCSI_CMND ... |
+- |
36| |
+---------------+---------------+---------------+---------------+
40| CDB ... |
+---------------+---------------+---------------+---------------+
x | (Data ...) |
+---------------+---------------+---------------+---------------+
The advantage of moving the Frame Check (if used) into the header would be
with respect to data placement especially in regard to potential overlays or
non-sequential data due to the freedom offered to SCSI. To assist in
delineating this transport as independent of SCSI a better name could be
SCSI Encapsulation Transport or SET. To indicate its use under IP, iSET.
The frame type could include FC or Parallel versions of SCSI to assist in a
more straight forward conversion.
Doug
> Santosh,
>
> Overlaps and out of order delivery and gaps are not forbidden by SAM . I
> think we have to go to T10 for that I can't see a good reason to do it. We
> have a good solution without asking for it . I can see large and
> important
> future application that will relay on overlaps and/or gaps and I am not
> going to foolishly do something to disable or harm their efficient
> implementation. I think that T10s philosophy of keeping the target master
> of the transfer and not limiting it in any way is too valuable to ignore.
>
> IMHO your request violates our charter without any good reason to support
> it.
>
> Julo
>
> Santosh Rao on 01/02/2001 03:53:13
>
> Please respond to Santosh Rao
>
> To: ips@ece.cmu.edu
> cc:
> Subject: Re: iSCSI : Holes in StatSN
>
>
>
>
> julian_satran@il.ibm.com wrote:
>
> > With a data sequence we may want to use a similar mechanism to ask for a
> > missed data block as soon as we see one of its successors or the status.
>
> Julian,
>
> The missing data PDU is missing due to either a header format
> error, header
> digest error or data digest error. [in all other cases, TCP ensures
> reliable
> delivery].
>
> In 2 of the above 3 cases, [header format error & header digest error] the
> initiator CANNOT do a safe interpretation of the PDU header. Without
> interpreting the PDU header, the initiator does NOT get the Initiator Task
> Tag. Any request to re-send a particular data PDU MUST be qualified by :
> I.T.T + missing_DataSN [+ T.T.T + CmdSN, optionally].
>
> Since I.T.T. cannot be reliably determined in 2 of the 3 cases, such a
> re-send request cannot be reliably achieved.
>
> The alternate proposal that was made should be considered in its place,
> which
> was to :
> - dis-allow overlapped data xfer's
> - initiators do a count check
> - a command level retry is performed at the iSCSI layer on detecting an
> underrun [due to a missing PDU].
>
> On several ocassions, requests from different people have been
> made on this
> list to dis-allow overlapped data xfers. Can a WG consensus be sought on
> this
> issue to see if the benefits of allowing overlapped data xfer's offset its
> complexities and justify its support ?
>
> Regards,
> Santosh
>
> - santoshr.vcf
>
>
>
>
From owner-ips@ece.cmu.edu Thu Feb 1 16:50:19 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA12304
for ; Thu, 1 Feb 2001 16:50:19 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11JRpK18937
for ips-outgoing; Thu, 1 Feb 2001 14:27:51 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11JRZZ18922
for ; Thu, 1 Feb 2001 14:27:35 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f11KbY065317;
Thu, 1 Feb 2001 12:37:35 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: ,
Subject: RE: iSCSI Data Integrity - Digests
Date: Thu, 1 Feb 2001 11:25:55 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
In-Reply-To: <0F31E5C394DAD311B60C00E029101A07041014C5@corpmx9.isus.emc.com>
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
David,
Once dealing with 32-bit wide memory accesses, adding and subtracting are
hardly difficult operations nor more difficult than that needed for CRC.
Adler32 does not involve division in most implementations and requires few
gates between states so minor pipe-lining easily handles any perceived
difficulty. You will notice that s2 does not feed into s1.
Here is the repetitive operation of adler32:
s1 += *buf++;
if (s1 >= BASE)
s1 -= BASE;
s2 += s1;
if (s2 >= BASE)
s2 -= BASE;
Doug
> > I am not sure I understand the difficulty imposed by adler32
> with respect
> to
> > hardware. Optimal assembly code looks different with about three
> > instructions per byte.
>
> I believe the concern is about how fast a hot-rodded ASIC can go.
> The arithmetic involved in CRCs doesn't have to cope with carry/borrow
> propagation in contrast to the arithmetic involved in the Adler-32
> modulus.
>
> --David
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA 01748
> +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500
> black_david@emc.com Mobile: +1 (978) 394-7754
> ---------------------------------------------------
>
>
From owner-ips@ece.cmu.edu Thu Feb 1 16:52:11 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA12371
for ; Thu, 1 Feb 2001 16:52:10 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11KLsW21595
for ips-outgoing; Thu, 1 Feb 2001 15:21:54 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from dns.LampreyNetworks.com (IDENT:root@mail.lampreynetworks.com [64.31.72.26])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11K5YZ20842
for ; Thu, 1 Feb 2001 15:05:34 -0500 (EST)
Received: from breinhold ([140.186.41.177])
by dns.LampreyNetworks.com (8.9.3/8.8.7) with SMTP id PAA30401;
Thu, 1 Feb 2001 15:07:33 -0500
From: "Barry Reinhold - Lamprey"
To: "Julian_Satran@Il. Ibm. Com"
Cc: "ISCSI"
Subject: ISCSI: Immediate data extending beyond a single PDU
Date: Thu, 1 Feb 2001 15:04:28 -0500
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Julian,
My understanding of your email in conjunction with the current draft is:
The initiator may send zero or more immediate data bytes in the CMD PDU, and
may send additional unsolicited DATA PDUs with further immediate data, with
the
last PDU having the F bit set. The total amount of immediate data to be
limited
by the value negotiated at login.
There are two issues I have with this:
First, the target does not know how much immediate data has being sent.
There
is no indication in the CMD frame to indicate that more immediate data is to
follow (whether or not the CMD frame contains any immediate data). The
initiator is free to send any amount, from zero up to the negotiated value.
Therefore, the target will likely send an R2T to the host at whatever
boundry
it believes is present - which may cause a retransmission of much or all of
the
immediate data.
Second, the additional unsolicited PDU's do not contain a target
task/transfer
tag. As such, in the mainline processing, you are perhaps forcing the target
to
look up the Initiator Task Tag in its list of active i/o's. This may exact a
heavy performance penalty as optimizing this lookup maybe cost-prohibitive.
The
only other scenarios in which the target had to perform this lookup was in
low-occurance error/abort conditions.
Are there negative implications to keeping immediate data to a single PDU?
-- Barry
julian_satran@il.ibm.com wrote:
> Barry,
>
> Immediate data is "attached to the command". If your unsolicited data (the
> so called initial burst) is larger that what you are ready to commit to a
> single PDU you can send several PDUs with the last having the F bit.
After
> that the target can decide that it wants more and send an R2T or send
> status.
>
> You are no supposed to send both immediate and separate PDU.
>
> I admit that the text is bit confusing (in 1.2.5) and there is an error
> also in the appendix.
> 2.7.1 talk about unsolicited data (not immediate) (the context is data
> PDUs).
>
> The new version of 1.2.5 now reads:
>
> Unsolicited data can be sent as part of an iSCSI command PDU
("immediate
> data") or an in separate iSCSI data PDUs. An initiator may send
> unsolicited data either as immediate (up to the negotiated maximum PDU
> size - DataPDULength - disconnect-reconnect mode page) or in a separate
> PDU sequence (up to the negotiated limit - FirstBurstSize -
> disconnect-reconnect mode page). All subsequent data have to be
> solicited. The maximum size of an individual data PDU or the
> immediate-part of the initial unsolicited burst as well as the initial
> burst size MAY be negotiated at login.
>
> Thanks,
> Julo
>
> "Barry Reinhold - Lamprey" on 29/01/2001
16:34:30
>
> Please respond to "Barry Reinhold - Lamprey"
>
> To: Julian Satran/Haifa/IBM@IBMIL
> cc: "Jon Sreekanth" , "James Smart"
>
> Subject:
>
> Issue:
>
> I am unclear on how to respond to an iSCSI SCSI command PDU when there is
> immediate data but not enough to meet the requirements of the IO operation
> as given in the expected data transfer length field. Should an R2T be
> issued
> or not?
>
> Background:
>
> Clause 2.7.1 in 03 reads:
>
> "2.7.1 F (Final) bit
>
> This bit is 1 for the last PDU of immediate data or the last PDU of a
> sequence answering a R2T."
>
> This suggests that immediate data can extend beyond the iSCSI SCSI CMD PDU
> (there is no final bit on the command PDU)but the closet thing we have to
a
> definition for immediate data, which is in clause 1.2.5 below, suggests
> that
> immediate data is limitted to the command PDU:
>
> "Outgoing SCSI data (initiator to target - user data or command
> parameters) will be sent as either solicited data or unsolicited
> data. Solicited data are sent in response to Ready To Transfer (R2T)
> PDUs. Unsolicited data can be part of an iSCSI command PDU
> ("immediate data") or an iSCSI data PDU. An initiator may send
> unsolicited data (immediate or in a separate PDU) up to the
> negotiated limit (initial burst size - mode page 02h). All subsequent
> data have to be solicited. The maximum size of an individual data
> PDU or the immediate-part of the initial unsolicited burst MAY be
> negotiated at login (maximum connect size - mode page 02h)."
>
> Unsolicited data is bounded in a number of ways, but most significant to
> this issue is in the description of the login key useR2t which states:
>
> "Note than only the first
> outgoing data item (either immediate data or a separate PDU) can be
> sent unsolicited by a R2T."
>
> Given the above it is unclear to me if the concept of immediate data is:
>
> 1. Write data that can be sent without an R2T (unsolicited write data)and
> starts in the command PDU.
> 2. Write data that is entirely contained within the command PDU. (my
> initial
> concept of immediate data)
> 3. The non-zero portion of data, in a possible multi-PDU write operation,
> that is contained in the iSCSI SCSI command PDU. Where the remaining PDUs
> of
> the write operation must be solicited with an R2T.
>
> Given the current definition of the Final bit and the current limits on
> unsolicited data it is unclear to me what a target should do when
receiving
> an iSCSI SCSI command PDU that has immediate data in it, but not
sufficient
> data to meet the expected data transfer length specificied in the command.
> Does an R2T go out of not?
>
> Barry Reinhold
> 603-659-0885
Barry Reinhold
603-659-0885
From owner-ips@ece.cmu.edu Thu Feb 1 16:53:04 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA12403
for ; Thu, 1 Feb 2001 16:53:04 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11KArW21089
for ips-outgoing; Thu, 1 Feb 2001 15:10:53 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from main.connectcom.net (ns1.advansys.com [204.247.22.3] (may be forged))
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11KA3Z21024
for ; Thu, 1 Feb 2001 15:10:03 -0500 (EST)
Received: from yp_portable (anubis.advansys.com [204.247.22.2]) by main.connectcom.net with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id 1DQGXJAB; Thu, 1 Feb 2001 12:09:59 -0800
From: "Y P Cheng"
To: "'Ips@Ece. Cmu. Edu'"
Subject: RE: iSCSI: CmdSN and Retry
Date: Thu, 1 Feb 2001 12:07:49 -0800
Message-ID: <011801c08c8a$a669c920$90c809c0@yp_portable.advansys.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3
Importance: Normal
In-Reply-To:
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Julo wrote:
> Except for sending the status - an executing command helds-up the LU queue
> and makes the "local" recovery simpler than clearing the LU queue and
> resending the commands.
Santosh wrote:
> This is correct, in the case where the target does NOT implement
> data/status recovery. i.e. Assume ordering is required, 2 commands , say,
> 1 & 2, executed in order at the target. Now, if 1 encountered a digest
error
> or format error at the initiator, and was re-sent with the "retry" bit
> AND the target were to NOT implement data/status recovery, it would result
> in target executing 1, 2, 1. This may be a problem and canot be addressed,
> unless iSCSI mandates data/status recovery.
I certainly understand the need of doing data/status recovery and the
argument of "local" recovery being simpler. However, this comes with heavy
cost of performance when pipelined design demands 100,000 IOs per second on
a network with long delay. For services and responses happening a few times
per second, it is OK to hold on the resources until we are certain the ACK
is returned. However, in the example above, after completing command 1 if a
target can't start command 2 until the status for 1 is ACK'ed, the wait can
be 100 milliseconds on a network with long delay. The wait make it
impossible to have large number of IOs in the pipeline. By mandating
data/status recovery in iSCSI, we change the pipelined command execution to
interlock handshakes. As I have said in the previous email, an initiator
will never send an command which depends on the success of a previous
command. This fact makes the pipeline execution in a target possible.
On a separate note, I really respect Santosh's fine-tooth analysis of the
iSCSI draft. But, in his arguments the fact that SCSI has been functional
for the last 20 years was badly ignored. The CmdSN, DataSN, and StatSN
allow iSCSI to detect missing PDUs and to quickly ask for retransmit. They
should not be used to enforce sequentiality to slow things down. SCSI
already has the semantics of ordered execution that requires the help of
CmdSN when multiple TCP connections are used. However, using StatSN to
mandate data/status retry pays a great performance price. Both overlapped
and out-of-order data transfers are allowed in SCSI (Check out the Modify
Data Pointer extended message). SCSI works fine without mandating
non-overlapping transfers or data/status recovery. Retry can be done in a
simple and clean manner without introducing complicated semantics for CmdSN,
DataSN, and StatSN. Note, if we must retry more than once in a million IOs,
something is wrong of the infrastructure. Therefore, let the pipeline flow
quickly and don't optimize the retry.
As long as we separate the TCP, iSCSI, and SCSI ULP layers cleanly -- for
which this WG has done a good job -- SCSI will continue to work. Without
wasting more bandwidth on this subject, I will be willing to discuss the
SCSI retry implementations with anyone offline.
From owner-ips@ece.cmu.edu Thu Feb 1 20:15:33 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA16130
for ; Thu, 1 Feb 2001 20:15:32 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11NHvf29212
for ips-outgoing; Thu, 1 Feb 2001 18:17:57 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11NHjZ29199
for ; Thu, 1 Feb 2001 18:17:45 -0500 (EST)
Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203])
by palrel1.hp.com (Postfix) with ESMTP
id 53C7EBA6; Thu, 1 Feb 2001 15:17:43 -0800 (PST)
Received: (from santoshr@localhost)
by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) id PAA28159;
Thu, 1 Feb 2001 15:17:38 -0800 (PST)
From: Santosh Rao
Message-Id: <200102012317.PAA28159@hpcuhe.cup.hp.com>
Subject: Re: iSCSI: CmdSN and Retry
To: ycheng@advansys.com (Y P Cheng)
Date: Thu, 1 Feb 2001 15:17:36 -0800 (PST)
Cc: ips@ece.cmu.edu ('Ips@Ece. Cmu. Edu')
In-Reply-To: <011801c08c8a$a669c920$90c809c0@yp_portable.advansys.com> from "Y P Cheng" at Feb 01, 2001 12:07:49 PM
X-Mailer: ELM [version 2.5 PL2]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> I certainly understand the need of doing data/status recovery and the
> argument of "local" recovery being simpler. However, this comes with
> heavy cost of performance when pipelined design demands 100,000 IOs per
> second on a network with long delay. For services and responses
> happening a few times per second, it is OK to hold on the resources
> until we are certain the ACK is returned. However, in the example
> above, after completing command 1 if a target can't start command 2
> until the status for 1 is ACK'ed, the wait can
> be 100 milliseconds on a network with long delay. The wait make it
> impossible to have large number of IOs in the pipeline. By mandating
> data/status recovery in iSCSI, we change the pipelined command
> execution to interlock handshakes.
> As I have said in the previous email, an initiator
> will never send an command which depends on the success of a previous
> command. This fact makes the pipeline execution in a target possible.
YP,
I agree with the above and have been saying the same thing in a seperate
thread. (See : "Command Ordering Proposal"
http://ips.pdl.cs.cmu.edu/mail/msg03171.html)
(The corner cases stated simply point out issues with the SNs as they exist
in the draft today.) The bigger question of why iSCSI is attempting to
provide ordering when its peers do not is still un-answered.
iSCSI is trying to enforce strict ordering which is not required due to the
following reasons :
1) Other SCSI Transport Protocols (FC, pSCSI) do not guarantee strict
ordering across commands.
2) Most (All ?) operating system SCSI Stacks enforce ordering above the
SCSI ULP and do not depend on the SCSI Stack to provide end-to-end
ordering across commands. (The exception to this is ordering of commands
followed by their aborts).
3) In the absence of (1) & (2) above, applications cannot depend on SCSI
transport protocols to provide strict end-to-end ordering across commands,
since they are transport agnostic and do not wish to base ordering
assumptions on the transport they operate on. (i.e. appln code enforces
ordering if it operates on FC and pSCSI and uses iSCSI ordering when it
runs above iSCSI !)
4) Error recovery scenarios needed to maintain ordering across transport
errors, resource allocation errors, errors at the target, digest errors
and format errors becomes extremely complex, stalls all pipelined
operations and in some cases cannot resolve the out-of-order situation
induced unless the target executes 1 command at a time and ONLY starts on
the next command after a StatSN ACK for the previous one.
5) Strict ordering results in performance penalties such as :
a) non-concurrent command execution at the targets.
b) stalled TCP connections in a session when a connection turns faulty.
c) all commands need to be stalled and re-initiated on any I/O error in an
attempt to maintain ordering.
d) Flow Control mechanisms like QUEUE FULL error recovery will change to
complete oscillating algorithms with the initiator completely stopping all
further I/Os until order is restored in the sequence they were originally
sent.
6) Strict ordering is not required in the majority if not all cases, since
most (all ?) O.S' do not enforce strict ordering or do anything
special in their error recovery to maintain strict ordering.
7) Several targets treat simple and ordered tags in a similar manner
without differentiating and providing strict ordering for ordered tag
commands.
8) Majority of the I/O traffic (if not all) is simple tag commands and all
this ordering and error recovery for ordering will impact performance when
ordering was never required in the first place.
9) The use of ordering within a SCSI stack is not prevalent. IN those few
places where it is known to be in use, this is a static characteristic of
that SCSI stack, and such stacks can use single-connection sessions to
achive their ordering.
IOW, the majority of stacks do NOT require strict ordering. Hence, iSCSI
should optimize for this most common case and NOT penalize it.
>
> On a separate note, I really respect Santosh's fine-tooth analysis of the
> iSCSI draft. But, in his arguments the fact that SCSI has been functional
> for the last 20 years was badly ignored.
YP, the corner cases simply bring out problems in the draft as they exist
today. I have been consistently raising the bigger question about why
iSCSI is attempting to differ from its peers in issues like ordering,
overlapped data xfer's, etc. SCSI has been functioning over the last 20
years without applns depending on SCSI stacks for strict ordering or
deploying un-used features like overlapped data xfers.
Applns will continue NOT to depend on scsi transports for the above since
they are written to be transport agnostic, and are based on lowest common
denominator support. (i.e. assume no ordering from O.S SCSI Stack).
> However, using StatSN to
> mandate data/status retry pays a great performance price.
Agreed.
> Both overlapped
> and out-of-order data transfers are allowed in SCSI (Check out the Modify
> Data Pointer extended message). SCSI works fine without mandating
> non-overlapping transfers or data/status recovery.
The Modify Data Pointers needs to be explicitly enabled through Mode
Select and I can't recollect seeing any instances of its usage.
As for FCP, overlapped data xfer's are not allowed. period.
Regards,
Santosh
--
#################################
Santosh Rao
Software Design Engineer,
HP, Cupertino.
email : santoshr@cup.hp.com
Phone : 408-447-3751
#################################
From owner-ips@ece.cmu.edu Thu Feb 1 20:15:42 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA16143
for ; Thu, 1 Feb 2001 20:15:41 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11NQvI29548
for ips-outgoing; Thu, 1 Feb 2001 18:26:57 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11NQ8Z29520
for ; Thu, 1 Feb 2001 18:26:08 -0500 (EST)
Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203])
by palrel1.hp.com (Postfix) with ESMTP
id 7CE165EA; Thu, 1 Feb 2001 15:26:06 -0800 (PST)
Received: (from santoshr@localhost)
by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) id PAA29083;
Thu, 1 Feb 2001 15:26:01 -0800 (PST)
From: Santosh Rao
Message-Id: <200102012326.PAA29083@hpcuhe.cup.hp.com>
Subject: Re: iSCSI: CmdSN and Retry
To: dotis@sanlight.net (Douglas Otis)
Date: Thu, 1 Feb 2001 15:26:01 -0800 (PST)
Cc: ycheng@advansys.com (Y P Cheng), ips@ece.cmu.edu ('Ips@Ece. Cmu. Edu')
In-Reply-To: from "Douglas Otis" at Feb 01, 2001 02:23:20 PM
X-Mailer: ELM [version 2.5 PL2]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> Unless the transport maintains order, there will be a requirement to wait
> for each command to complete to assure the sequence.
Doug,
The majority of SCSI stacks today DO function as stated above.
The layer above ULP enforces the ordering and awaits completion of 1 I/O
prior to posting the subsequent I/O , PROVIDED those 2 I/Os required
ordering to be maintained.
> This makes
> compliance to SCSI impossible and not a means of improving the pipe-line.
Are you saying that the majority of stacks today are not
SCSI compliant (?)
YP raises a valid concern in that if strict ordering is expected from the
transport, it should be capable of providing this 100 %. (98% strict
ordering is'nt good enough.). To attempt to provide 100% guarantee of
strict ordering is going to prevent concurrent I/O processing at the
target at a session level (NOT even a LUN level, which is the granularity
where one would expect ordering, if at all !).
Regards,
Santosh
--
#################################
Santosh Rao
Software Design Engineer,
HP, Cupertino.
email : santoshr@cup.hp.com
Phone : 408-447-3751
#################################
From owner-ips@ece.cmu.edu Thu Feb 1 20:15:45 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA16141
for ; Thu, 1 Feb 2001 20:15:40 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11MPsX27187
for ips-outgoing; Thu, 1 Feb 2001 17:25:54 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11MP4Z27131
for ; Thu, 1 Feb 2001 17:25:04 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f11NYx065445;
Thu, 1 Feb 2001 15:35:03 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: "Y P Cheng" , "'Ips@Ece. Cmu. Edu'"
Subject: RE: iSCSI: CmdSN and Retry
Date: Thu, 1 Feb 2001 14:23:20 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
In-Reply-To: <011801c08c8a$a669c920$90c809c0@yp_portable.advansys.com>
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Y.P.
Any TCP bandwidth is limited to .93 MSS/(RTT * loss_rate^-2) so your
assumption of one in a million at 100 milli-seconds limits at Fast Ethernet
rates. There will be significant overhead on long pipes dealing with TCP
handshakes and any additional handshake will not add significantly to this
resource overhead. The more immediate these handshakes, the less resources
consumed with respect to any needed replays to satisfy the transport. I do
not agree procrastination that depends on rediscovery of indexes unrelated
to these handshakes to be more effective in allowing pipe-line execution.
Unless the transport maintains order, there will be a requirement to wait
for each command to complete to assure the sequence. This makes compliance
to SCSI impossible and not a means of improving the pipe-line. An immediate
handshake within the transport layer dealing with digest errors is the best
means of improving performance. Reliance on the SCSI tag to recover from a
failure deduced from a dropped transport sequence is not a clean separation
of layers. The amount of resources saved by deleting this positive
relationship is dwarfed by the window size required for such a fat pipe.
Doug
Should each IO represent
> Julo wrote:
> > Except for sending the status - an executing command helds-up
> the LU queue
> > and makes the "local" recovery simpler than clearing the LU queue and
> > resending the commands.
> Santosh wrote:
> > This is correct, in the case where the target does NOT implement
> > data/status recovery. i.e. Assume ordering is required, 2
> commands , say,
> > 1 & 2, executed in order at the target. Now, if 1 encountered a digest
> error
> > or format error at the initiator, and was re-sent with the "retry" bit
> > AND the target were to NOT implement data/status recovery, it
> would result
> > in target executing 1, 2, 1. This may be a problem and canot be
> addressed,
> > unless iSCSI mandates data/status recovery.
>
> I certainly understand the need of doing data/status recovery and the
> argument of "local" recovery being simpler. However, this comes
> with heavy
> cost of performance when pipelined design demands 100,000 IOs per
> second on
> a network with long delay. For services and responses happening
> a few times
> per second, it is OK to hold on the resources until we are certain the ACK
> is returned. However, in the example above, after completing
> command 1 if a
> target can't start command 2 until the status for 1 is ACK'ed,
> the wait can
> be 100 milliseconds on a network with long delay. The wait make it
> impossible to have large number of IOs in the pipeline. By mandating
> data/status recovery in iSCSI, we change the pipelined command
> execution to
> interlock handshakes. As I have said in the previous email, an initiator
> will never send an command which depends on the success of a previous
> command. This fact makes the pipeline execution in a target possible.
>
> On a separate note, I really respect Santosh's fine-tooth analysis of the
> iSCSI draft. But, in his arguments the fact that SCSI has been functional
> for the last 20 years was badly ignored. The CmdSN, DataSN, and StatSN
> allow iSCSI to detect missing PDUs and to quickly ask for
> retransmit. They
> should not be used to enforce sequentiality to slow things down. SCSI
> already has the semantics of ordered execution that requires the help of
> CmdSN when multiple TCP connections are used. However, using StatSN to
> mandate data/status retry pays a great performance price. Both overlapped
> and out-of-order data transfers are allowed in SCSI (Check out the Modify
> Data Pointer extended message). SCSI works fine without mandating
> non-overlapping transfers or data/status recovery. Retry can be done in a
> simple and clean manner without introducing complicated semantics
> for CmdSN,
> DataSN, and StatSN. Note, if we must retry more than once in a
> million IOs,
> something is wrong of the infrastructure. Therefore, let the
> pipeline flow
> quickly and don't optimize the retry.
>
> As long as we separate the TCP, iSCSI, and SCSI ULP layers cleanly -- for
> which this WG has done a good job -- SCSI will continue to work. Without
> wasting more bandwidth on this subject, I will be willing to discuss the
> SCSI retry implementations with anyone offline.
>
>
From owner-ips@ece.cmu.edu Thu Feb 1 20:15:58 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA16163
for ; Thu, 1 Feb 2001 20:15:58 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11NAtG28903
for ips-outgoing; Thu, 1 Feb 2001 18:10:55 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from main.connectcom.net (ns1.advansys.com [204.247.22.3] (may be forged))
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11NAgZ28889
for ; Thu, 1 Feb 2001 18:10:42 -0500 (EST)
Received: from yp_portable (anubis.advansys.com [204.247.22.2]) by main.connectcom.net with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id 1DQGXJHB; Thu, 1 Feb 2001 15:10:40 -0800
From: "Y P Cheng"
To: "Douglas Otis" ,
"'Ips@Ece. Cmu. Edu'"
Subject: RE: iSCSI: CmdSN and Retry
Date: Thu, 1 Feb 2001 15:08:31 -0800
Message-ID: <012a01c08ca3$e4c2bf60$90c809c0@yp_portable.advansys.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3
Importance: Normal
In-Reply-To:
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Doug,
I think you have misunderstood what I said about error recovery in different
layers. I specifically have stated that we should separate the three
layers: 1) Transport layer provides in order delivery on Internet, 2) iSCSI
keeps the in order among multiple connections, and 3) SCSI has its own in
order command semantics. Frame retransmit and congestion control belong to
the transport. Digest error retransmit belongs to iSCSI layer. Finally,
SCSI itself has done command level recovery very well for the last 20 years.
We should mix iSCSI and SCSI retries. I want to be able to pipe 100,000
frames down the wire without letting retry and ACKs getting into the way.
In my previous writings, I have stated that for TCP to support 10Gb/100Ms
network with a huge reassembly buffer, message framing and TCP/RDMA must be
used. Whether the maximum TCP bandwidth supports such a high IO rate is
beyond the Charters of this WG. I have no problem in using SCTP, if this is
what it would take for the 10Gb/100Ms network.
Y.P.
> Any TCP bandwidth is limited to .93 MSS/(RTT * loss_rate^-2) so your
> assumption of one in a million at 100 milli-seconds limits at
> Fast Ethernet
> rates. There will be significant overhead on long pipes dealing with TCP
> handshakes and any additional handshake will not add significantly to this
> resource overhead. The more immediate these handshakes, the less
> resources
> consumed with respect to any needed replays to satisfy the
> transport. I do
> not agree procrastination that depends on rediscovery of indexes unrelated
> to these handshakes to be more effective in allowing pipe-line execution.
> Unless the transport maintains order, there will be a requirement to wait
> for each command to complete to assure the sequence. This makes
> compliance
> to SCSI impossible and not a means of improving the pipe-line.
> An immediate
> handshake within the transport layer dealing with digest errors
> is the best
> means of improving performance. Reliance on the SCSI tag to
> recover from a
> failure deduced from a dropped transport sequence is not a clean
> separation
> of layers. The amount of resources saved by deleting this positive
> relationship is dwarfed by the window size required for such a fat pipe.
>
> Doug
From owner-ips@ece.cmu.edu Thu Feb 1 20:16:09 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA16174
for ; Thu, 1 Feb 2001 20:16:08 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f11NFu529143
for ips-outgoing; Thu, 1 Feb 2001 18:15:56 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from main.connectcom.net (ns1.advansys.com [204.247.22.3] (may be forged))
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11NFWZ29070
for ; Thu, 1 Feb 2001 18:15:32 -0500 (EST)
Received: from yp_portable (anubis.advansys.com [204.247.22.2]) by main.connectcom.net with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id 1DQGXJHK; Thu, 1 Feb 2001 15:15:31 -0800
From: "Y P Cheng"
To: "Douglas Otis" ,
"'Ips@Ece. Cmu. Edu'"
Subject: RE: iSCSI: CmdSN and Retry
Date: Thu, 1 Feb 2001 15:13:21 -0800
Message-ID: <012b01c08ca4$91c7f5e0$90c809c0@yp_portable.advansys.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3
Importance: Normal
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> Frame retransmit and congestion control belong to the transport.
> Digest error retransmit belongs to iSCSI layer. Finally, SCSI
> itself has done command level recovery very well for the last 20
> years. We should mix iSCSI and SCSI retries.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Should NOT. My apology.
From owner-ips@ece.cmu.edu Thu Feb 1 21:23:36 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA17148
for ; Thu, 1 Feb 2001 21:23:35 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f120auZ02189
for ips-outgoing; Thu, 1 Feb 2001 19:36:56 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from palrel3.hp.com (palrel3.hp.com [156.153.255.226])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f120aLZ02167
for ; Thu, 1 Feb 2001 19:36:21 -0500 (EST)
Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203])
by palrel3.hp.com (Postfix) with ESMTP
id BFEA7976; Thu, 1 Feb 2001 16:36:19 -0800 (PST)
Received: (from santoshr@localhost)
by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) id QAA07788;
Thu, 1 Feb 2001 16:36:15 -0800 (PST)
From: Santosh Rao
Message-Id: <200102020036.QAA07788@hpcuhe.cup.hp.com>
Subject: Re: iSCSI: CmdSN and Retry
To: dotis@sanlight.net (Douglas Otis)
Date: Thu, 1 Feb 2001 16:36:14 -0800 (PST)
Cc: santoshr@cup.hp.com (Santosh Rao), ips@ece.cmu.edu (ips)
In-Reply-To: from "Douglas Otis" at Feb 01, 2001 04:15:26 PM
X-Mailer: ELM [version 2.5 PL2]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> An extra handshake that requires a small delay should there
> be a non-sequential event would occur orders of magnitude less often than
> similar waits for TCP. As this must be a rare event, the wait will be
> insignificant.
Doug,
The non-sequential event may be rare. However, the handshake should occur
on EVERY I/O to safeguard against that rare event. [if 100% ordering is
attempted.].
Strict ordering of 2 commands, say, A followed by B, imply that :
A target cannot execute B until the initiator acknowledges Status
PDU for A. (This implies sequential operation on I/Os on the task set
similar to untagged targets, only greater, because this behaviour is per
session and not even per LUN.)
Since iSCSI applies strict ordering to all commands (well, all non-0 CmdSN
commands), the above behaviour will be required on all I/Os within a
session.
Regards,
Santosh
--
#################################
Santosh Rao
Software Design Engineer,
HP, Cupertino.
email : santoshr@cup.hp.com
Phone : 408-447-3751
#################################
From owner-ips@ece.cmu.edu Thu Feb 1 21:26:37 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA17165
for ; Thu, 1 Feb 2001 21:26:29 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f120Ju801570
for ips-outgoing; Thu, 1 Feb 2001 19:19:56 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f120JbZ01551
for ; Thu, 1 Feb 2001 19:19:37 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f121R5065531;
Thu, 1 Feb 2001 17:27:05 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: "Santosh Rao"
Cc: "Y P Cheng" , "'Ips@Ece. Cmu. Edu'"
Subject: RE: iSCSI: CmdSN and Retry
Date: Thu, 1 Feb 2001 16:15:26 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
In-Reply-To: <200102012326.PAA29083@hpcuhe.cup.hp.com>
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Santosh,
TCP mandates sequential delivery. As was often stated on this reflector,
framing would be to place data payloads but would not impact the sequence of
any operation. An extra handshake that requires a small delay should there
be a non-sequential event would occur orders of magnitude less often than
similar waits for TCP. As this must be a rare event, the wait will be
insignificant. Assurance of sequential delivery improves reliability and
does not require command-by-command pacing to provide expected SCSI
operation that otherwise is easily done with parallel SCSI configurations.
The buffer size will be nominally one packet larger to handle such extra
handshake and again insignificant. I see nothing to justify complexity
offered by convoluted recovery for a digest error. SCTP offers a means of
delivering objects out-of-sequence and then perhaps flagging which objects
can violate SCSI's sense of ordering could be done to help facilitate long
pipes. If this ever becomes a significant factor, the bandwidth will be so
small as to be of little concern anyway.
Doug
> > Unless the transport maintains order, there will be a
> requirement to wait
> > for each command to complete to assure the sequence.
>
> Doug,
>
> The majority of SCSI stacks today DO function as stated above.
> The layer above ULP enforces the ordering and awaits completion of 1 I/O
> prior to posting the subsequent I/O , PROVIDED those 2 I/Os required
> ordering to be maintained.
>
> > This makes
> > compliance to SCSI impossible and not a means of improving the
> pipe-line.
> Are you saying that the majority of stacks today are not
> SCSI compliant (?)
>
> YP raises a valid concern in that if strict ordering is expected from the
> transport, it should be capable of providing this 100 %. (98% strict
> ordering is'nt good enough.). To attempt to provide 100% guarantee of
> strict ordering is going to prevent concurrent I/O processing at the
> target at a session level (NOT even a LUN level, which is the granularity
> where one would expect ordering, if at all !).
>
> Regards,
> Santosh
>
>
>
> --
> #################################
> Santosh Rao
> Software Design Engineer,
> HP, Cupertino.
> email : santoshr@cup.hp.com
> Phone : 408-447-3751
> #################################
>
From owner-ips@ece.cmu.edu Thu Feb 1 23:21:33 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA19804
for ; Thu, 1 Feb 2001 23:21:32 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f121dxV04304
for ips-outgoing; Thu, 1 Feb 2001 20:39:59 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f121dBZ04283
for ; Thu, 1 Feb 2001 20:39:11 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f122l6065597;
Thu, 1 Feb 2001 18:47:06 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: "Santosh Rao"
Cc: "ips"
Subject: RE: iSCSI: CmdSN and Retry
Date: Thu, 1 Feb 2001 17:35:27 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
In-Reply-To: <200102020036.QAA07788@hpcuhe.cup.hp.com>
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Santosh,
1. iSCSI is based on TCP, a protocol that mandates sequential delivery.
2. TCP provides insignificant bandwidth if there is a high error rate.
3. Digest errors will happen at the 16-bit checksum undetected error rate.
4. Higher latency of networks makes such command-by-command constraints less
effective than a transport that assures sequencing.
5. There are many cases (not most) where order is significant. You could
make a better case for ensuring sequencing at the transport level than
negligible benefits derived from non-sequential processing of commands
following a digest error.
> Doug,
>
> The non-sequential event may be rare. However, the handshake should occur
> on EVERY I/O to safeguard against that rare event. [if 100% ordering is
> attempted.].
The only potential delay would be with respect to any skew between
connections. If the PDUs are delivered in order, as they would be normally,
then there would be zero delay in the processing of the information. The
pending handshake would only require the sender to retain a structure that
relates the sequence value to the PDU sent. This handshake is trivial,
already within the structures, and only requires the return of an additional
packet before this structure can be removed. Yes every I/O would be
examined to ensure its sequence but the same check is made to acknowledge
now. Normally this results in no extra operation or delay.
> Strict ordering of 2 commands, say, A followed by B, imply that :
>
> A target cannot execute B until the initiator acknowledges Status
> PDU for A. (This implies sequential operation on I/Os on the task set
> similar to untagged targets, only greater, because this behaviour is per
> session and not even per LUN.)
As status should enjoy the same independent assurance of delivery, SCSI
would not be impacted by a transport error as you imply nor would there be a
need to execute command-status-command-status. If you could not assure
reliable sequential delivery, then should such become important, then indeed
you would be required to execute command-status-command-status. The benefit
from using the handshake would be to prevent this requirement. Digest
errors will happen at the 16-bit checksum undetected error rate. We are
taking about a flea of a flea with respect to any loss of performance.
> Since iSCSI applies strict ordering to all commands (well, all non-0 CmdSN
> commands), the above behaviour will be required on all I/Os within a
> session.
This is the claim but a claim that can not be assured due to the lack of
integrity in the current handshake.
Doug
> Regards,
> Santosh
>
>
> --
> #################################
> Santosh Rao
> Software Design Engineer,
> HP, Cupertino.
> email : santoshr@cup.hp.com
> Phone : 408-447-3751
> #################################
>
From owner-ips@ece.cmu.edu Thu Feb 1 23:58:50 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA20443
for ; Thu, 1 Feb 2001 23:58:48 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f122t2E06868
for ips-outgoing; Thu, 1 Feb 2001 21:55:02 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from dns.LampreyNetworks.com (IDENT:root@mail.lampreynetworks.com [64.31.72.26])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f11K5YZ20842
for ; Thu, 1 Feb 2001 15:05:34 -0500 (EST)
Received: from breinhold ([140.186.41.177])
by dns.LampreyNetworks.com (8.9.3/8.8.7) with SMTP id PAA30401;
Thu, 1 Feb 2001 15:07:33 -0500
From: "Barry Reinhold - Lamprey"
To: "Julian_Satran@Il. Ibm. Com"
Cc: "ISCSI"
Subject: ISCSI: Immediate data extending beyond a single PDU
Date: Thu, 1 Feb 2001 15:04:28 -0500
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Julian,
My understanding of your email in conjunction with the current draft is:
The initiator may send zero or more immediate data bytes in the CMD PDU, and
may send additional unsolicited DATA PDUs with further immediate data, with
the
last PDU having the F bit set. The total amount of immediate data to be
limited
by the value negotiated at login.
There are two issues I have with this:
First, the target does not know how much immediate data has being sent.
There
is no indication in the CMD frame to indicate that more immediate data is to
follow (whether or not the CMD frame contains any immediate data). The
initiator is free to send any amount, from zero up to the negotiated value.
Therefore, the target will likely send an R2T to the host at whatever
boundry
it believes is present - which may cause a retransmission of much or all of
the
immediate data.
Second, the additional unsolicited PDU's do not contain a target
task/transfer
tag. As such, in the mainline processing, you are perhaps forcing the target
to
look up the Initiator Task Tag in its list of active i/o's. This may exact a
heavy performance penalty as optimizing this lookup maybe cost-prohibitive.
The
only other scenarios in which the target had to perform this lookup was in
low-occurance error/abort conditions.
Are there negative implications to keeping immediate data to a single PDU?
-- Barry
julian_satran@il.ibm.com wrote:
> Barry,
>
> Immediate data is "attached to the command". If your unsolicited data (the
> so called initial burst) is larger that what you are ready to commit to a
> single PDU you can send several PDUs with the last having the F bit.
After
> that the target can decide that it wants more and send an R2T or send
> status.
>
> You are no supposed to send both immediate and separate PDU.
>
> I admit that the text is bit confusing (in 1.2.5) and there is an error
> also in the appendix.
> 2.7.1 talk about unsolicited data (not immediate) (the context is data
> PDUs).
>
> The new version of 1.2.5 now reads:
>
> Unsolicited data can be sent as part of an iSCSI command PDU
("immediate
> data") or an in separate iSCSI data PDUs. An initiator may send
> unsolicited data either as immediate (up to the negotiated maximum PDU
> size - DataPDULength - disconnect-reconnect mode page) or in a separate
> PDU sequence (up to the negotiated limit - FirstBurstSize -
> disconnect-reconnect mode page). All subsequent data have to be
> solicited. The maximum size of an individual data PDU or the
> immediate-part of the initial unsolicited burst as well as the initial
> burst size MAY be negotiated at login.
>
> Thanks,
> Julo
>
> "Barry Reinhold - Lamprey" on 29/01/2001
16:34:30
>
> Please respond to "Barry Reinhold - Lamprey"
>
> To: Julian Satran/Haifa/IBM@IBMIL
> cc: "Jon Sreekanth" , "James Smart"
>
> Subject:
>
> Issue:
>
> I am unclear on how to respond to an iSCSI SCSI command PDU when there is
> immediate data but not enough to meet the requirements of the IO operation
> as given in the expected data transfer length field. Should an R2T be
> issued
> or not?
>
> Background:
>
> Clause 2.7.1 in 03 reads:
>
> "2.7.1 F (Final) bit
>
> This bit is 1 for the last PDU of immediate data or the last PDU of a
> sequence answering a R2T."
>
> This suggests that immediate data can extend beyond the iSCSI SCSI CMD PDU
> (there is no final bit on the command PDU)but the closet thing we have to
a
> definition for immediate data, which is in clause 1.2.5 below, suggests
> that
> immediate data is limitted to the command PDU:
>
> "Outgoing SCSI data (initiator to target - user data or command
> parameters) will be sent as either solicited data or unsolicited
> data. Solicited data are sent in response to Ready To Transfer (R2T)
> PDUs. Unsolicited data can be part of an iSCSI command PDU
> ("immediate data") or an iSCSI data PDU. An initiator may send
> unsolicited data (immediate or in a separate PDU) up to the
> negotiated limit (initial burst size - mode page 02h). All subsequent
> data have to be solicited. The maximum size of an individual data
> PDU or the immediate-part of the initial unsolicited burst MAY be
> negotiated at login (maximum connect size - mode page 02h)."
>
> Unsolicited data is bounded in a number of ways, but most significant to
> this issue is in the description of the login key useR2t which states:
>
> "Note than only the first
> outgoing data item (either immediate data or a separate PDU) can be
> sent unsolicited by a R2T."
>
> Given the above it is unclear to me if the concept of immediate data is:
>
> 1. Write data that can be sent without an R2T (unsolicited write data)and
> starts in the command PDU.
> 2. Write data that is entirely contained within the command PDU. (my
> initial
> concept of immediate data)
> 3. The non-zero portion of data, in a possible multi-PDU write operation,
> that is contained in the iSCSI SCSI command PDU. Where the remaining PDUs
> of
> the write operation must be solicited with an R2T.
>
> Given the current definition of the Final bit and the current limits on
> unsolicited data it is unclear to me what a target should do when
receiving
> an iSCSI SCSI command PDU that has immediate data in it, but not
sufficient
> data to meet the expected data transfer length specificied in the command.
> Does an R2T go out of not?
>
> Barry Reinhold
> 603-659-0885
Barry Reinhold
603-659-0885
From owner-ips@ece.cmu.edu Fri Feb 2 08:21:18 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id IAA07614
for ; Fri, 2 Feb 2001 08:21:17 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f12BkAM04139
for ips-outgoing; Fri, 2 Feb 2001 06:46:10 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f12BjjZ04125
for ; Fri, 2 Feb 2001 06:45:46 -0500 (EST)
Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22])
by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id MAA81086
for ; Fri, 2 Feb 2001 12:45:39 +0100
From: julian_satran@il.ibm.com
Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237])
by d12relay01.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id MAA145004
for ; Fri, 2 Feb 2001 12:45:39 +0100
Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569E7.0040994B ; Fri, 2 Feb 2001 12:45:35 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: ips@ece.cmu.edu
Message-ID:
Date: Fri, 2 Feb 2001 13:41:10 +0200
Subject: Re: ISCSI: Immediate data extending beyond a single PDU
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Barry,
There is a misunderstanding:
You can send EITHER immediate data OR a sequence of unsolicited PDUs.
Immediate data was conceived for short writes - for which an initiator does
not want to do
two socket operations or 2 HBA operations.
Larger unsolicited bursts can be sent with a sequence and then we felt that
you won't gain too much by having the first one glued to the command.
Julo
"Barry Reinhold - Lamprey" on 01/02/2001 22:04:28
Please respond to "Barry Reinhold - Lamprey"
To: Julian Satran/Haifa/IBM@IBMIL
cc: "ISCSI"
Subject: ISCSI: Immediate data extending beyond a single PDU
Julian,
My understanding of your email in conjunction with the current draft is:
The initiator may send zero or more immediate data bytes in the CMD PDU,
and
may send additional unsolicited DATA PDUs with further immediate data, with
the
last PDU having the F bit set. The total amount of immediate data to be
limited
by the value negotiated at login.
There are two issues I have with this:
First, the target does not know how much immediate data has being sent.
There
is no indication in the CMD frame to indicate that more immediate data is
to
follow (whether or not the CMD frame contains any immediate data). The
initiator is free to send any amount, from zero up to the negotiated value.
Therefore, the target will likely send an R2T to the host at whatever
boundry
it believes is present - which may cause a retransmission of much or all of
the
immediate data.
Second, the additional unsolicited PDU's do not contain a target
task/transfer
tag. As such, in the mainline processing, you are perhaps forcing the
target
to
look up the Initiator Task Tag in its list of active i/o's. This may exact
a
heavy performance penalty as optimizing this lookup maybe cost-prohibitive.
The
only other scenarios in which the target had to perform this lookup was in
low-occurance error/abort conditions.
Are there negative implications to keeping immediate data to a single PDU?
-- Barry
julian_satran@il.ibm.com wrote:
> Barry,
>
> Immediate data is "attached to the command". If your unsolicited data
(the
> so called initial burst) is larger that what you are ready to commit to a
> single PDU you can send several PDUs with the last having the F bit.
After
> that the target can decide that it wants more and send an R2T or send
> status.
>
> You are no supposed to send both immediate and separate PDU.
>
> I admit that the text is bit confusing (in 1.2.5) and there is an error
> also in the appendix.
> 2.7.1 talk about unsolicited data (not immediate) (the context is data
> PDUs).
>
> The new version of 1.2.5 now reads:
>
> Unsolicited data can be sent as part of an iSCSI command PDU
("immediate
> data") or an in separate iSCSI data PDUs. An initiator may send
> unsolicited data either as immediate (up to the negotiated maximum PDU
> size - DataPDULength - disconnect-reconnect mode page) or in a
separate
> PDU sequence (up to the negotiated limit - FirstBurstSize -
> disconnect-reconnect mode page). All subsequent data have to be
> solicited. The maximum size of an individual data PDU or the
> immediate-part of the initial unsolicited burst as well as the initial
> burst size MAY be negotiated at login.
>
> Thanks,
> Julo
>
> "Barry Reinhold - Lamprey" on 29/01/2001
16:34:30
>
> Please respond to "Barry Reinhold - Lamprey"
>
> To: Julian Satran/Haifa/IBM@IBMIL
> cc: "Jon Sreekanth" , "James Smart"
>
> Subject:
>
> Issue:
>
> I am unclear on how to respond to an iSCSI SCSI command PDU when there is
> immediate data but not enough to meet the requirements of the IO
operation
> as given in the expected data transfer length field. Should an R2T be
> issued
> or not?
>
> Background:
>
> Clause 2.7.1 in 03 reads:
>
> "2.7.1 F (Final) bit
>
> This bit is 1 for the last PDU of immediate data or the last PDU of a
> sequence answering a R2T."
>
> This suggests that immediate data can extend beyond the iSCSI SCSI CMD
PDU
> (there is no final bit on the command PDU)but the closet thing we have to
a
> definition for immediate data, which is in clause 1.2.5 below, suggests
> that
> immediate data is limitted to the command PDU:
>
> "Outgoing SCSI data (initiator to target - user data or command
> parameters) will be sent as either solicited data or unsolicited
> data. Solicited data are sent in response to Ready To Transfer (R2T)
> PDUs. Unsolicited data can be part of an iSCSI command PDU
> ("immediate data") or an iSCSI data PDU. An initiator may send
> unsolicited data (immediate or in a separate PDU) up to the
> negotiated limit (initial burst size - mode page 02h). All subsequent
> data have to be solicited. The maximum size of an individual data
> PDU or the immediate-part of the initial unsolicited burst MAY be
> negotiated at login (maximum connect size - mode page 02h)."
>
> Unsolicited data is bounded in a number of ways, but most significant to
> this issue is in the description of the login key useR2t which states:
>
> "Note than only the first
> outgoing data item (either immediate data or a separate PDU) can be
> sent unsolicited by a R2T."
>
> Given the above it is unclear to me if the concept of immediate data is:
>
> 1. Write data that can be sent without an R2T (unsolicited write data)and
> starts in the command PDU.
> 2. Write data that is entirely contained within the command PDU. (my
> initial
> concept of immediate data)
> 3. The non-zero portion of data, in a possible multi-PDU write operation,
> that is contained in the iSCSI SCSI command PDU. Where the remaining PDUs
> of
> the write operation must be solicited with an R2T.
>
> Given the current definition of the Final bit and the current limits on
> unsolicited data it is unclear to me what a target should do when
receiving
> an iSCSI SCSI command PDU that has immediate data in it, but not
sufficient
> data to meet the expected data transfer length specificied in the
command.
> Does an R2T go out of not?
>
> Barry Reinhold
> 603-659-0885
Barry Reinhold
603-659-0885
From owner-ips@ece.cmu.edu Fri Feb 2 08:24:45 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id IAA07696
for ; Fri, 2 Feb 2001 08:24:44 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f12BLAl03441
for ips-outgoing; Fri, 2 Feb 2001 06:21:10 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f12BKbZ03429
for ; Fri, 2 Feb 2001 06:20:38 -0500 (EST)
Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22])
by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id MAA57876
for ; Fri, 2 Feb 2001 12:20:30 +0100
From: julian_satran@il.ibm.com
Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237])
by d12relay01.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id MAA282230
for ; Fri, 2 Feb 2001 12:20:30 +0100
Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569E7.003E4C1F ; Fri, 2 Feb 2001 12:20:27 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: ips@ece.cmu.edu
Message-ID:
Date: Fri, 2 Feb 2001 13:16:01 +0200
Subject: Re: iSCSI : Holes in StatSN
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
comments in text - Julo
Santosh Rao on 01/02/2001 20:51:35
Please respond to Santosh Rao
To: IPS Reflector
cc:
Subject: Re: iSCSI : Holes in StatSN
julian_satran@il.ibm.com wrote:
> The sack will specify a range to be filled. The initiator will revert to
> bulk as soon as it has no holes.
Julian,
Once a StatSN hole is created, it is never filled. [since subsequent
"retry"
of the command results in a different StatSN being used.]. However, there
is
currently no association available to determine when it is safe to
acknowledge that hole [which happens when the initiator switches back to
bulk
ACK scheme.]
Given the above, the initiator needs to maintain a list of Initiator Task
Tags for which StatSN holes were encountered (i.e. Status PDU was not
received). It then needs to dequeue elements from this list when the
"retry"
command completes or the command is aborted. Once all elements are off this
list, it can revert to the bulk StatSN ACK mechanism.
> A target can also resend after a timeout or when seen the same StatSN on
a
> NOP.
Not sure what is implied here. Surely, we are not attempting a re-transmit
functionality akin to TCP re-transmit at the iSCSI layer for Status PDUs
(?)
> The initiator will query after
> a long silence with a NOP (not longer than the SCSI timeout -:))
This is intriguing. Are you suggesting that on an I/O timeout, the
initiator
send a NOP-OUT to request a re-transmit of the Status PDU ? How does the
initiator know one of the Data PDUs also did not time out ?
+++ I the status PDU (alone or with a data block) there is a the last data
block sequence number
This is how holes in data will be detected (the same as with sum of
data-lengths versus expected-residual that you are advocating)
(An I/O timeout also implies that the O.S. expects a response back without
any further time being spent on the I/O. I/O timeouts cause the initiator
to
abort the I/O and error the I/O up the stack. )
Regards,
Santosh
> Santosh Rao on 01/02/2001 02:57:25
>
> Please respond to Santosh Rao
>
> To:
> cc: ips@ece.cmu.edu
> Subject: Re: iSCSI : Holes in StatSN
>
> Julian,
>
> Once the selective StatSN ACK mechanism kicks in, how is the initiator to
> revert to the bulk StatSN ACK ? (i.e. when/how does an initiator realize
> that
> the hole is filled ?) Or, does it only use selective StatSN ACK from
there
> on
> ?
>
> The difficulty lies in the fact that a hole created will never be filled
> since "retry" will result in target sending back a subsequent Status PDU
> with
> a different StatSN. However, the initiator does not know when to safely
> claim
> that the hole is filled (by sending a bulk StatSN ACK), since there is no
> way
> to detect this.
>
> Regards,
> Santosh
>
- santoshr.vcf
From owner-ips@ece.cmu.edu Fri Feb 2 13:10:53 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id NAA19524
for ; Fri, 2 Feb 2001 13:10:50 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f12FPGs11377
for ips-outgoing; Fri, 2 Feb 2001 10:25:16 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from main.connectcom.net (ns1.advansys.com [204.247.22.3] (may be forged))
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f12FOQZ11342
for ; Fri, 2 Feb 2001 10:24:26 -0500 (EST)
Received: from yp_portable (anubis.advansys.com [204.247.22.2]) by main.connectcom.net with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id 11CPQC7C; Fri, 2 Feb 2001 07:24:25 -0800
From: "Y P Cheng"
To: "Michael Krause"
Cc:
Subject: RE: iSCSI Data Integrity - Digests
Date: Fri, 2 Feb 2001 07:22:25 -0800
Message-ID: <000001c08d2b$f33327a0$90c809c0@yp_portable.advansys.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
Importance: Normal
In-Reply-To: <5.0.0.25.2.20010201073752.02800e88@hpindlm.cup.hp.com>
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> The continual assertion that TCP implementations are sacrosanct and thus
> not cannot be modified to support iSCSI combined with a tendency to make
> everything an option should someone object quite frankly leads one to
> question whether iSCSI will suffer the same 10-year growing pains /
> interoperability problems FC suffered and thus seriously raises a
> question of its viability within the industry.
Mike, if iSCSI is deployed through the use of an adapter which collapses all
three layers, SCSI, iSCSI, and TCP, into a single set of microcode, then,
you got a new implementation of TCP anyway. All those sacrosanct talks that
forbid changes of TCP implementations do not apply to an iSCSI adapter which
at login time will exchange text parameters with each other to deploy new
TCP options. Everything you wanted can be inside an iSCSI adapter.
However, the right place to talk about what you want from TCP is in the
end2end group, not here, as so many people repeatedly pointed out to me. :-)
Y.P.
From owner-ips@ece.cmu.edu Fri Feb 2 14:18:57 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA23293
for ; Fri, 2 Feb 2001 14:18:55 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f12EFGq08782
for ips-outgoing; Fri, 2 Feb 2001 09:15:16 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [168.159.208.52])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f12EF6Z08776
for ; Fri, 2 Feb 2001 09:15:06 -0500 (EST)
Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2650.21)
id <1DCB7626>; Fri, 2 Feb 2001 09:15:00 -0500
Message-ID: <0F31E5C394DAD311B60C00E029101A07041014D0@corpmx9.isus.emc.com>
From: Black_David@emc.com
To: ips@ece.cmu.edu
Subject: iSCSI error recovery
Date: Fri, 2 Feb 2001 09:14:59 -0500
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: text/plain
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
There's been a lot of discussion on the list about this
topic, including the proper use of CmdSN and StatSN
to respond to errors including those detected by CRC.
I hesitate to call consensus on any of this discussion
because it's based on a version of the draft (-03) that
is some combination of incomplete, incorrect, and/or
difficult to comprehend in this area. Based on this
and the length/detail of the discussion, I suspect
that many people on the list are not completely aware
of the issues here, and calling consensus when a large
segment of the WG may not understand the issues is
unlikely to be productive.
The next version of the draft (-04) SHOULD contain a
much better section on error recovery with explanations
and examples that make it significantly easier to
understand. Consensus calls on error recovery will not
be made before it appears.
Meanwhile, a few comments on ongoing issues:
- If a CRC error occurs, trying to use any data covered
by that CRC is generally not a good idea in the
absence of other measures (e.g., an error-correcting
code covering the field(s) of interest).
- Given some of the CRC discussion, I think the conclusion
in Orlando to have separate header and data CRCs is
NOT the rough consensus of the WG. We need to go
back to a requirements discussion on this rather than
debating exactly where to put the CRCs. Would those
envisioning middleboxes/gateways/etc. that would
benefit from this sort of CRC separation please post
short use cases/descriptions indicating the basic
functionality of the box and which fields it needs
to change (let's do this with reference to the header
layout in -03 rather than subsequent changes)? As
part of the use case/description, please explain
how/why Fibre Channel's single CRC covering both
the frame header and data causes problems/difficulties.
- Mandating the use of IPsec or TLS solely to handle
CRC-level integrity issues is overkill.
Thanks,
--David
---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA 01748
+1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500
black_david@emc.com Mobile: +1 (978) 394-7754
---------------------------------------------------
From owner-ips@ece.cmu.edu Fri Feb 2 14:27:33 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA23625
for ; Fri, 2 Feb 2001 14:27:30 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f12H4Jw15352
for ips-outgoing; Fri, 2 Feb 2001 12:04:19 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from main.connectcom.net (ns1.advansys.com [204.247.22.3] (may be forged))
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f12H3GZ15313
for ; Fri, 2 Feb 2001 12:03:16 -0500 (EST)
Received: from yp_portable (anubis.advansys.com [204.247.22.2]) by main.connectcom.net with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id 11CPQC03; Fri, 2 Feb 2001 09:03:15 -0800
From: "Y P Cheng"
To:
Subject: RE: iSCSI error recovery
Date: Fri, 2 Feb 2001 09:01:16 -0800
Message-ID: <000801c08d39$c17b27e0$90c809c0@yp_portable.advansys.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
Importance: Normal
In-Reply-To: <0F31E5C394DAD311B60C00E029101A07041014D0@corpmx9.isus.emc.com>
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> Meanwhile, a few comments on ongoing issues:
> - Given some of the CRC discussion, I think the conclusion
> in Orlando to have separate header and data CRCs is
> NOT the rough consensus of the WG. We need to go
> back to a requirements discussion on this rather than
> debating exactly where to put the CRCs. Would those
> envisioning middleboxes/gateways/etc. that would
> benefit from this sort of CRC separation please post
> short use cases/descriptions indicating the basic
> functionality of the box and which fields it needs
> to change (let's do this with reference to the header
> layout in -03 rather than subsequent changes)? As
> part of the use case/description, please explain
> how/why Fibre Channel's single CRC covering both
> the frame header and data causes problems/difficulties.
With a single CRC covering both the frame header and payload, when a bad
frame is thrown away which is the only one in a sequence, the only way for
detecting the missing frame is timeout. With separate CRCs for header and
payload, if the header CRC is still good, one can quickly inform the sender
about the error. However, for iSCSI, the need for separate CRCs is not so
obvious in this context. This is because the TCP header is still valid.
Hence, without relying on timeout, one can use on the TCP header to identify
the sender. Needless to say, any attempt to use the contents covered by the
bad CRC is a cardinal sin for folks in storage industry.
Y.P. Cheng, CTO, ConnectCom Solutions Corp.
From owner-ips@ece.cmu.edu Fri Feb 2 14:34:50 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA23823
for ; Fri, 2 Feb 2001 14:34:49 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f12HNHG16200
for ips-outgoing; Fri, 2 Feb 2001 12:23:17 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from mercury.Sun.COM (mercury.Sun.COM [192.9.25.1])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f12HMpZ16188
for ; Fri, 2 Feb 2001 12:22:51 -0500 (EST)
Received: from phys-ha10nwka.ebay.sun.com ([129.150.151.210])
by mercury.Sun.COM (8.9.3+Sun/8.9.3) with ESMTP id JAA07813
for ; Fri, 2 Feb 2001 09:22:49 -0800 (PST)
Received: from ebay.sun.com (d-nwk02-144-195 [129.150.144.195])
by phys-ha10nwka.ebay.sun.com (8.8.8+Sun/8.8.8/ENSMAIL,v2.0) with ESMTP id JAA26626
for ; Fri, 2 Feb 2001 09:22:48 -0800 (PST)
Message-ID: <3A7AED08.8DEEB65C@ebay.sun.com>
Date: Fri, 02 Feb 2001 09:23:20 -0800
From: David Robinson
Organization: Sun Microsystems, Inc.
X-Mailer: Mozilla 4.75 [en] (Win98; U)
X-Accept-Language: en
MIME-Version: 1.0
To: ips@ece.cmu.edu
Subject: Re: iSCSI Data Integrity - Digests
References: <000001c08d2b$f33327a0$90c809c0@yp_portable.advansys.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Y P Cheng wrote:
> Mike, if iSCSI is deployed through the use of an adapter which collapses all
> three layers, SCSI, iSCSI, and TCP, into a single set of microcode, then,
> you got a new implementation of TCP anyway. All those sacrosanct talks that
> forbid changes of TCP implementations do not apply to an iSCSI adapter which
> at login time will exchange text parameters with each other to deploy new
> TCP options. Everything you wanted can be inside an iSCSI adapter.
> However, the right place to talk about what you want from TCP is in the
> end2end group, not here, as so many people repeatedly pointed out to me. :-)
YP,
While I understand and can appreciate all the tricks that can be done
with
a merged stack that avoids the implementation layer, I strongly object
to
this WG allowing standard options that change the standard TCP wire
protocol.
Even if both ends agree to do the "special" processing, other nodes and
protocols generally share these wires and "special" non-standard options
can have nasty indirect effects. If your custom adapter does things
that
are not changes in the protocol, like always putting an iSCSI request in
a
single segment which allows the other end to fast-path processing, is
an acceptable implementation trick as either end not "knowing" about the
trick will function correctly, although maybe not optimally.
Any changes to the standard TCP must go through end2end, this WG MUST
NOT
allow options that circumvent that process.
-David
From owner-ips@ece.cmu.edu Fri Feb 2 17:03:46 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA01440
for ; Fri, 2 Feb 2001 17:03:45 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f12JYKG21618
for ips-outgoing; Fri, 2 Feb 2001 14:34:20 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from palrel1.hp.com (palrel1.hp.com [156.153.255.242])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f12JXFZ21580
for ; Fri, 2 Feb 2001 14:33:19 -0500 (EST)
Received: from hpcuhe.cup.hp.com (hpcuhe.cup.hp.com [15.0.80.203])
by palrel1.hp.com (Postfix) with ESMTP id 7376A135C
for ; Fri, 2 Feb 2001 11:26:20 -0800 (PST)
Received: from cup.hp.com (santoshr@hpindhhm.cup.hp.com [15.8.80.197])
by hpcuhe.cup.hp.com (8.9.3 (PHNE_18979)/8.9.3 SMKit7.02) with ESMTP id LAA17754
for ; Fri, 2 Feb 2001 11:26:11 -0800 (PST)
Message-ID: <3A7B0A5A.47909B63@cup.hp.com>
Date: Fri, 02 Feb 2001 11:28:27 -0800
From: Santosh Rao
Organization: Hewlett Packard, Cupertino.
X-Mailer: Mozilla 4.7 [en] (X11; U; HP-UX B.11.00 9000/778)
X-Accept-Language: en
MIME-Version: 1.0
To: ips@ece.cmu.edu
Subject: Re: iSCSI error recovery
References: <000801c08d39$c17b27e0$90c809c0@yp_portable.advansys.com>
Content-Type: multipart/mixed;
boundary="------------69D6AAD56FE38184889DA736"
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
This is a multi-part message in MIME format.
--------------69D6AAD56FE38184889DA736
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
YP,
Y P Cheng wrote:
> With a single CRC covering both the frame header and payload, when a bad
> frame is thrown away which is the only one in a sequence, the only way for
> detecting the missing frame is timeout.
Not true for a Data PDU. An I/O timeout only occurs when the Status PDU is
discarded due to a CRC error [or format error]. Discarding a Data PDU [due to a
CRC error or format error] results in I/O underrun, [a terminology used to
describe initiator receiving less data than sent by target] and can be detected
by a count check . [using a DataSN based count as described by Julian in his
last clarification on this issue.]
If the Data PDU also contained the Status within, then discarding such a PDU
does cause an I/O timeout.
Regards,
Santosh
--------------69D6AAD56FE38184889DA736
Content-Type: text/x-vcard; charset=us-ascii;
name="santoshr.vcf"
Content-Description: Card for Santosh Rao
Content-Disposition: attachment;
filename="santoshr.vcf"
Content-Transfer-Encoding: 7bit
begin:vcard
n:Rao;Santosh
tel;work:408-447-3751
x-mozilla-html:FALSE
org:Hewlett Packard, Cupertino.;SISL
adr:;;19420, Homestead Road, M\S 43LN, ;Cupertino.;CA.;95014.;USA.
version:2.1
email;internet:santoshr@cup.hp.com
title:Software Design Engineer
x-mozilla-cpt:;21088
fn:Santosh Rao
end:vcard
--------------69D6AAD56FE38184889DA736--
From owner-ips@ece.cmu.edu Fri Feb 2 17:05:48 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA01494
for ; Fri, 2 Feb 2001 17:05:47 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f12K2La22853
for ips-outgoing; Fri, 2 Feb 2001 15:02:21 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f12K1fZ22827
for ; Fri, 2 Feb 2001 15:01:42 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f12LBf066810;
Fri, 2 Feb 2001 13:11:42 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: ,
Subject: RE: iSCSI error recovery
Date: Fri, 2 Feb 2001 12:00:03 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
In-Reply-To: <0F31E5C394DAD311B60C00E029101A07041014D0@corpmx9.isus.emc.com>
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
David,
Although further refinement of how the initiator is expected to use or not
use StatSN will not modify complexity involved in recovering from a digest
error after some undefined timeout of expected status. The initiator is to
guess a sequence hole belongs to some pending SCSI tag? It is to decide if
it should apply a CmdSN based on this lack of status for a retry? Is this
complexity is to allow non-sequential processing of commands irrespective of
delivery failures? As you can see, I remain skeptical.
With respect to CRC or Checksums, you could also include a finer subdivision
than just header and data. To be more explicit about potential options,
there could be static and dynamic fields where to assist handshakes being
debated, a separate checksum for these handshakes would have advantages.
You could then add routing information as yet another field if this
information is to be modified by middle tier equipment. And finally, the
tag, CDB and related data and associated vectors. So from my perspective
here is a breakdown:
1) Dynamic fields regarding handshakes. (Hopefully obsoleting retry scheme.)
2) Routing fields (LUN).
3) Static fields (All else).
4) SCSI Cmnd (attributes), CDB, Data.
Where would you wish to see a separation between 1:2, 2:3, or 3:4 or a
single Checksum?
Doug
> There's been a lot of discussion on the list about this
> topic, including the proper use of CmdSN and StatSN
> to respond to errors including those detected by CRC.
> I hesitate to call consensus on any of this discussion
> because it's based on a version of the draft (-03) that
> is some combination of incomplete, incorrect, and/or
> difficult to comprehend in this area. Based on this
> and the length/detail of the discussion, I suspect
> that many people on the list are not completely aware
> of the issues here, and calling consensus when a large
> segment of the WG may not understand the issues is
> unlikely to be productive.
>
> The next version of the draft (-04) SHOULD contain a
> much better section on error recovery with explanations
> and examples that make it significantly easier to
> understand. Consensus calls on error recovery will not
> be made before it appears.
>
> Meanwhile, a few comments on ongoing issues:
> - If a CRC error occurs, trying to use any data covered
> by that CRC is generally not a good idea in the
> absence of other measures (e.g., an error-correcting
> code covering the field(s) of interest).
> - Given some of the CRC discussion, I think the conclusion
> in Orlando to have separate header and data CRCs is
> NOT the rough consensus of the WG. We need to go
> back to a requirements discussion on this rather than
> debating exactly where to put the CRCs. Would those
> envisioning middleboxes/gateways/etc. that would
> benefit from this sort of CRC separation please post
> short use cases/descriptions indicating the basic
> functionality of the box and which fields it needs
> to change (let's do this with reference to the header
> layout in -03 rather than subsequent changes)? As
> part of the use case/description, please explain
> how/why Fibre Channel's single CRC covering both
> the frame header and data causes problems/difficulties.
> - Mandating the use of IPsec or TLS solely to handle
> CRC-level integrity issues is overkill.
>
> Thanks,
> --David
>
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA 01748
> +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500
> black_david@emc.com Mobile: +1 (978) 394-7754
> ---------------------------------------------------
>
>
From owner-ips@ece.cmu.edu Fri Feb 2 18:19:37 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA04563
for ; Fri, 2 Feb 2001 18:19:36 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f12KiOK24795
for ips-outgoing; Fri, 2 Feb 2001 15:44:24 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from main.connectcom.net (ns1.advansys.com [204.247.22.3] (may be forged))
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f12KhJZ24741
for ; Fri, 2 Feb 2001 15:43:20 -0500 (EST)
Received: from yp_portable (anubis.advansys.com [204.247.22.2]) by main.connectcom.net with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id 11CPQDL2; Fri, 2 Feb 2001 12:43:17 -0800
From: "Y P Cheng"
To: "David Robinson" ,
Subject: RE: iSCSI Data Integrity - Digests
Date: Fri, 2 Feb 2001 12:41:09 -0800
Message-ID: <000601c08d58$79254ce0$90c809c0@yp_portable.advansys.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
Importance: Normal
In-Reply-To: <3A7AED08.8DEEB65C@ebay.sun.com>
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> While I understand and can appreciate all the tricks that can be done
> with a merged stack that avoids the implementation layer, I strongly
> object to this WG allowing standard options that change the standard
> TCP wire protocol. Even if both ends agree to do the "special"
> processing, other nodes and protocols generally share these
> wires and "special" non-standard options can have nasty indirect
> effects. If your custom adapter does things that are not changes
> in the protocol, like always putting an iSCSI request in a
> single segment which allows the other end to fast-path processing, is
> an acceptable implementation trick as either end not "knowing" about the
> trick will function correctly, although maybe not optimally.
> Any changes to the standard TCP must go through end2end, this WG MUST
> NOT allow options that circumvent that process.
> -David
David,
Each time when I touched this area, I always felt like walking around a
snake pit. It is neither my intent to discuss changes of TCP wire protocol
in this WG nor my desire to make such changes. I've been always carefully
in using the words "TCP implementation." In fact, after reading a lots of
RFC's, I have come to the conclusion that there are ways within TCP that
would allow iSCSI work well in 10GB/100Ms Network. The slow TCP transport
problem is in the existing implementations that do not have all the new TCP
options.
For example, you can have 16-bit TCP window-size or 32-bit. Using 16-bit
window size, there in no way we can keep the 10Gb/100Ms pipe full. But, the
use of 32-bit window-size does not change the TCP protocol. Another
example, an iSCSI adapter can always send each iSCSI Command PDU in a
separate TCP segment. In so doing, URG pointer is perfect for marking
message boundary. While the gurus of TCP can quickly point out that it is
impossible for URG pointer to mark the SCSI command boundary in the BSD
implementation, there is nothing preventing two iSCSI adapters from using
URG pointer when they send command PDUs to each other. In fact, whether two
iSCSI adapters using URG pointer in their exchanges, IMHO, is even outside
the jurisdiction of this WG. Because the use of URG pointer does not
violate TCP protocol. It is just that old TCP implementations not able to
keep command/status PDUs in separate segments can not benefit from URG
pointer.
Therefore, while an iSCSI adapter can interoperate with a client running in
TCP of BSD 4.3, it should also be able to interoperate with another
state-of-art iSCSI adapter which is much more intelligent. The difference
in their interoperation will be decided at login with text messages. I
think we don't have to limit ourselves to stone age tools if electrical
drill is already invented. Please note that I am not asking the invention
of new TCP options here.
Y.P.
From owner-ips@ece.cmu.edu Fri Feb 2 19:24:28 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id TAA06689
for ; Fri, 2 Feb 2001 19:24:28 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f12M7Mj28086
for ips-outgoing; Fri, 2 Feb 2001 17:07:22 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from main.connectcom.net (ns1.advansys.com [204.247.22.3] (may be forged))
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f12M6OZ28053
for ; Fri, 2 Feb 2001 17:06:24 -0500 (EST)
Received: from yp_portable (anubis.advansys.com [204.247.22.2]) by main.connectcom.net with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id 11CPQDP8; Fri, 2 Feb 2001 14:06:21 -0800
From: "Y P Cheng"
To: "Santosh Rao" ,
Subject: RE: iSCSI error recovery
Date: Fri, 2 Feb 2001 14:04:09 -0800
Message-ID: <002401c08d64$1137f5e0$90c809c0@yp_portable.advansys.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
Importance: Normal
In-Reply-To: <3A7B0A5A.47909B63@cup.hp.com>
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> > With a single CRC covering both the frame header and payload, when a bad
> > frame is thrown away which is the only one in a sequence, the
> > only way for detecting the missing frame is timeout.
>
> Not true for a Data PDU. An I/O timeout only occurs when the Status PDU is
> discarded due to a CRC error [or format error]. Discarding a Data
> PDU [due to a CRC error or format error] results in I/O underrun,
> [a terminology used to describe initiator receiving less data than
> sent by target] and can be detected by a count check .
> [using a DataSN based count as described by Julian in his
> last clarification on this issue.]
>
> If the Data PDU also contained the Status within, then discarding
> such a PDU does cause an I/O timeout.
> Santosh
Lets try a little more specific:
1. Bad command PDU Thrown away by target Initiator Timeout
2. Bad write data Thrown away by target Target Timeout
3. Bad read data Thrown away by initiator Initiator data underrun
4. Bad status PDU Thrown away by initiator Initiator Timeout
In case 1, in receiving another command frame, the target detects a missing
CmdSN and decides to wait for a while and then asks initiator to resend.
In case 2, the target can detect a missing DataSN except the very last one.
Case 3 is the specific case you are refrering to.
In case 4, in receiving another status frame, the initiator detects a
missing StatSN and decides to wait for a while and then asks target to
resend.
Y.P.
From owner-ips@ece.cmu.edu Fri Feb 2 20:32:46 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA08815
for ; Fri, 2 Feb 2001 20:32:45 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f12N7NG00384
for ips-outgoing; Fri, 2 Feb 2001 18:07:23 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f12N6iZ00359
for ; Fri, 2 Feb 2001 18:06:44 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f130GZ066952;
Fri, 2 Feb 2001 16:16:36 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: "Y P Cheng" ,
"David Robinson" ,
Subject: RE: iSCSI Data Integrity - Digests
Date: Fri, 2 Feb 2001 15:04:57 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
In-Reply-To: <000601c08d58$79254ce0$90c809c0@yp_portable.advansys.com>
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> While the gurus of TCP can quickly point out that it is
> impossible for URG pointer to mark the SCSI command boundary in the BSD
> implementation, there is nothing preventing two iSCSI adapters from using
> URG pointer when they send command PDUs to each other. In fact,
> whether two
> iSCSI adapters using URG pointer in their exchanges, IMHO, is even outside
> the jurisdiction of this WG. Because the use of URG pointer does not
> violate TCP protocol. It is just that old TCP implementations not able to
> keep command/status PDUs in separate segments can not benefit from URG
> pointer.
I see a spark of life still lives in hopes of redefining the Urgent pointer.
A problem you will see is this pointer is only 16 bits. As it is defined to
coalesce should new points become defined, it takes a small backlog in the
TCP stack to create a pointer that can not reach a point being defined at
the API. To use the Urgent pointer as a message marker, this issue has been
decided and, in short, your conclusion about the ability to use the urgent
pointer in the manner you wish is re-inventing TCP and not to be done. If
you wish such non-standard implementations of TCP, it clearly is outside the
benefits of this WG.
Doug
> Therefore, while an iSCSI adapter can interoperate with a client
> running in
> TCP of BSD 4.3, it should also be able to interoperate with another
> state-of-art iSCSI adapter which is much more intelligent. The difference
> in their interoperation will be decided at login with text messages. I
> think we don't have to limit ourselves to stone age tools if electrical
> drill is already invented. Please note that I am not asking the invention
> of new TCP options here.
>
> Y.P.
From owner-ips@ece.cmu.edu Fri Feb 2 20:38:52 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA08880
for ; Fri, 2 Feb 2001 20:38:51 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f1307SM02225
for ips-outgoing; Fri, 2 Feb 2001 19:07:28 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from main.connectcom.net (ns1.advansys.com [204.247.22.3] (may be forged))
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f1306YZ02192
for ; Fri, 2 Feb 2001 19:06:34 -0500 (EST)
Received: from yp_portable (anubis.advansys.com [204.247.22.2]) by main.connectcom.net with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id 11CPQDXN; Fri, 2 Feb 2001 16:06:32 -0800
From: "Y P Cheng"
To: "Douglas Otis" ,
"David Robinson" ,
Subject: RE: iSCSI Data Integrity - Digests
Date: Fri, 2 Feb 2001 16:04:06 -0800
Message-ID: <003801c08d74$d3189420$90c809c0@yp_portable.advansys.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
Importance: Normal
In-Reply-To:
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> I see a spark of life still lives in hopes of redefining the
> Urgent pointer. A problem you will see is this pointer is only 16 bits.
> As it is defined to coalesce should new points become defined,
> it takes a small backlog in the TCP stack to create a pointer that
> can not reach a point being defined at the API. To use the Urgent
> pointer as a message marker, this issue has been
> decided and, in short, your conclusion about the ability to use the urgent
> pointer in the manner you wish is re-inventing TCP and not to be done. If
> you wish such non-standard implementations of TCP, it clearly is
> outside the benefits of this WG.
>
> Doug
Hi, Doug:
The Charters of this WG prohibits us discussing any change of TCP protocol
and I was not talking about changing any TCP protocol. I am quite familiar
with the history why urgent pointer wouldn't work as so many TCP gurus were
too quickly to point out. This WG does not recommend the use of urgent
pointer nor does it endorse the placement of a command/status PDU in a
separate TCP segment.
However, for microcode that running in my iSCSI interface chip with my TCP
acceleration implementation, I can place each command/status PDU in a
separate TCP segment without violating the TCP protocol. Similarly, in the
manner I understood about TCP if I turn on the urgent pointer when I send
the PDU to another iSCSI chip of which I came to know at login, I don't
believe I have altered TCP.
Some people in this WG might not like what could be done by each iSCSI
interface chip designer. However, this method of doing something special in
a SCSI device not available by others gives one a unique advantage and is a
common practice by the SCSI industry for many years. I will not be
surprised that each iSCSI interface chip will have its own bag of tricks to
accelerate its performance and still totally confirm the iSCSI
specifications. This is how the technology world is. :-)
Y.P.
From owner-ips@ece.cmu.edu Sun Feb 4 07:02:57 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id HAA07977
for ; Sun, 4 Feb 2001 07:02:57 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f14APO310078
for ips-outgoing; Sun, 4 Feb 2001 05:25:24 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f14AOb610052
for ; Sun, 4 Feb 2001 05:24:38 -0500 (EST)
Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22])
by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id LAA77560
for ; Sun, 4 Feb 2001 11:24:31 +0100
From: julian_satran@il.ibm.com
Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237])
by d12relay01.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id LAA81714
for ; Sun, 4 Feb 2001 11:24:31 +0100
Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569E9.00392C23 ; Sun, 4 Feb 2001 11:24:28 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: ips@ece.cmu.edu
Message-ID:
Date: Sun, 4 Feb 2001 12:19:57 +0200
Subject: RE: ISCSI: Immediate data extending beyond a single PDU
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Barry,
I think that the current draft does not allow you to accept immediate but
not unsolicited.
You can either allow unsolicited (including immediate) or request always
R2T.
And yes - if the expected length is larger that what you sent as
unsolicited (even if the unsolicited was less than the limit) the target is
supposed to send R2T (there is only one unsolicited burst per command).
The limit for immediate data is implicit - the max-PDU length and is
less-then-or-equal the first burst.
Your proposal is interesting but I doubt that it is very useful.
Unsolicited data help avoid an RTT on writes but require resources to be
allocated (that is why a target may not accept them).
Immediate data is an "optimized" form of unsolicited data in which the
command and data are collapsed to one unit. Nevertheless the immediate data
are unsolicited and require resources to be allocated ahead of time. If you
have larger amounts to send than what fits into one PDU the "collapse gain"
is minimal and does not justify the added complexity of 2 options.
Regards,
Julo
"Barry Reinhold" on 02/02/2001 21:06:29
Please respond to "Barry Reinhold"
To: Julian Satran/Haifa/IBM@IBMIL
cc:
Subject: RE: ISCSI: Immediate data extending beyond a single PDU
Julian,
I agree with the conceptual idea of immediate data, but further
refinement
is required. As a receiver, if I have allowed immediate data, but not
unsolicited data, and I get a write command with an expected data transfer
length that is greater then the amount of data in the command PDU, do I
send
an R2T or not (or from the sender's point of view does he wait for an R2T
to
complete the data transfer?) The draft is not clear on this, yet it is an
externally visible behavior that needs to be defined for interoperability.
I believe that what we need here is a refinement of two distinct concepts,
that of immediate data and that of unsolicited data. Here is my take at it:
1. Immediate data - Data that is sent within a single iSCSI SCSI command
PDU. Immediate data shall be fully contained within a single iSCSI command
PDU, and shall be limitted by the negotiated value of ImmediateData. The
size of Immediate data is not subject to the limits of FirstBurst.
2. Unsolicited data - Data that may be transfered to a target without an
explicit R2T being sent to request its transfer. Unsolicited data is sent
in
iSCSI SCSI data PDUs in which the target transfer tag is invalid. The
maximum amount of unsolicited data is negotiated between the initiator and
target through the FirstBurst key. If unsolicited data is supported by the
target, upon receipt of a write command, the target shall not send an R2T
until an iSCSI Data PDU with the Final bit set has been received. After
reception of an iSCSI Data PDU with the Final bit set R2Ts shall be sent by
the target to request transmission of any additional data associated with
this command. The initiator shall not send any additional unsolicited data
for this command.
Interaction between Immediate data and Unsolicited data -
Immediate data and Unsolicited data are features that may be selected
independently of one another. Support of either is optional.
Note: Based on the current specification of FirstBurst, support for
unsolicited data is not optional. One must select between a range of
1-65536. I think we would want to make this optional and indicate non
support for unsolicited data by setting FirstBurst to 0.
>-----Original Message-----
>From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
>julian_satran@il.ibm.com
>Sent: Friday, February 02, 2001 6:41 AM
>To: ips@ece.cmu.edu
>Subject: Re: ISCSI: Immediate data extending beyond a single PDU
>
>
>
>
>Barry,
>
>There is a misunderstanding:
>
>You can send EITHER immediate data OR a sequence of unsolicited PDUs.
>
>Immediate data was conceived for short writes - for which an initiator
does
>not want to do
>two socket operations or 2 HBA operations.
>
>Larger unsolicited bursts can be sent with a sequence and then we felt
that
>you won't gain too much by having the first one glued to the command.
>
>
>Julo
>
>"Barry Reinhold - Lamprey" on 01/02/2001
22:04:28
>
>Please respond to "Barry Reinhold - Lamprey"
>
>To: Julian Satran/Haifa/IBM@IBMIL
>cc: "ISCSI"
>Subject: ISCSI: Immediate data extending beyond a single PDU
>
>
>
>
>Julian,
>
>My understanding of your email in conjunction with the current draft is:
>
>The initiator may send zero or more immediate data bytes in the CMD PDU,
>and
>may send additional unsolicited DATA PDUs with further immediate data,
with
>the
>last PDU having the F bit set. The total amount of immediate data to be
>limited
>by the value negotiated at login.
>
>There are two issues I have with this:
>
>First, the target does not know how much immediate data has being sent.
>There
>is no indication in the CMD frame to indicate that more immediate data is
>to
>follow (whether or not the CMD frame contains any immediate data). The
>initiator is free to send any amount, from zero up to the negotiated
value.
>Therefore, the target will likely send an R2T to the host at whatever
>boundry
>it believes is present - which may cause a retransmission of much or all
of
>the
>immediate data.
>
>Second, the additional unsolicited PDU's do not contain a target
>task/transfer
>tag. As such, in the mainline processing, you are perhaps forcing the
>target
>to
>look up the Initiator Task Tag in its list of active i/o's. This may exact
>a
>heavy performance penalty as optimizing this lookup maybe
cost-prohibitive.
>The
>only other scenarios in which the target had to perform this lookup was in
>low-occurance error/abort conditions.
>
>Are there negative implications to keeping immediate data to a single PDU?
>
>-- Barry
>
>
>julian_satran@il.ibm.com wrote:
>
>> Barry,
>>
>> Immediate data is "attached to the command". If your unsolicited data
>(the
>> so called initial burst) is larger that what you are ready to commit to
a
>> single PDU you can send several PDUs with the last having the F bit.
>After
>> that the target can decide that it wants more and send an R2T or send
>> status.
>>
>> You are no supposed to send both immediate and separate PDU.
>>
>> I admit that the text is bit confusing (in 1.2.5) and there is an error
>> also in the appendix.
>> 2.7.1 talk about unsolicited data (not immediate) (the context is data
>> PDUs).
>>
>> The new version of 1.2.5 now reads:
>>
>> Unsolicited data can be sent as part of an iSCSI command PDU
>("immediate
>> data") or an in separate iSCSI data PDUs. An initiator may send
>> unsolicited data either as immediate (up to the negotiated maximum
PDU
>> size - DataPDULength - disconnect-reconnect mode page) or in a
>separate
>> PDU sequence (up to the negotiated limit - FirstBurstSize -
>> disconnect-reconnect mode page). All subsequent data have to be
>> solicited. The maximum size of an individual data PDU or the
>> immediate-part of the initial unsolicited burst as well as the
initial
>> burst size MAY be negotiated at login.
>>
>> Thanks,
>> Julo
>>
>> "Barry Reinhold - Lamprey" on 29/01/2001
>16:34:30
>>
>> Please respond to "Barry Reinhold - Lamprey"
>>
>> To: Julian Satran/Haifa/IBM@IBMIL
>> cc: "Jon Sreekanth" , "James Smart"
>>
>> Subject:
>>
>> Issue:
>>
>> I am unclear on how to respond to an iSCSI SCSI command PDU when there
is
>> immediate data but not enough to meet the requirements of the IO
>operation
>> as given in the expected data transfer length field. Should an R2T be
>> issued
>> or not?
>>
>> Background:
>>
>> Clause 2.7.1 in 03 reads:
>>
>> "2.7.1 F (Final) bit
>>
>> This bit is 1 for the last PDU of immediate data or the last PDU of a
>> sequence answering a R2T."
>>
>> This suggests that immediate data can extend beyond the iSCSI SCSI CMD
>PDU
>> (there is no final bit on the command PDU)but the closet thing we have
to
>a
>> definition for immediate data, which is in clause 1.2.5 below, suggests
>> that
>> immediate data is limitted to the command PDU:
>>
>> "Outgoing SCSI data (initiator to target - user data or command
>> parameters) will be sent as either solicited data or unsolicited
>> data. Solicited data are sent in response to Ready To Transfer (R2T)
>> PDUs. Unsolicited data can be part of an iSCSI command PDU
>> ("immediate data") or an iSCSI data PDU. An initiator may send
>> unsolicited data (immediate or in a separate PDU) up to the
>> negotiated limit (initial burst size - mode page 02h). All subsequent
>> data have to be solicited. The maximum size of an individual data
>> PDU or the immediate-part of the initial unsolicited burst MAY be
>> negotiated at login (maximum connect size - mode page 02h)."
>>
>> Unsolicited data is bounded in a number of ways, but most significant to
>> this issue is in the description of the login key useR2t which states:
>>
>> "Note than only the first
>> outgoing data item (either immediate data or a separate PDU) can be
>> sent unsolicited by a R2T."
>>
>> Given the above it is unclear to me if the concept of immediate data is:
>>
>> 1. Write data that can be sent without an R2T (unsolicited write
data)and
>> starts in the command PDU.
>> 2. Write data that is entirely contained within the command PDU. (my
>> initial
>> concept of immediate data)
>> 3. The non-zero portion of data, in a possible multi-PDU write
operation,
>> that is contained in the iSCSI SCSI command PDU. Where the remaining
PDUs
>> of
>> the write operation must be solicited with an R2T.
>>
>> Given the current definition of the Final bit and the current limits on
>> unsolicited data it is unclear to me what a target should do when
>receiving
>> an iSCSI SCSI command PDU that has immediate data in it, but not
>sufficient
>> data to meet the expected data transfer length specificied in the
>command.
>> Does an R2T go out of not?
>>
>> Barry Reinhold
>> 603-659-0885
>
>Barry Reinhold
>603-659-0885
>
>
>
>
From owner-ips@ece.cmu.edu Sun Feb 4 07:05:04 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id HAA07988
for ; Sun, 4 Feb 2001 07:05:04 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f149vOt09534
for ips-outgoing; Sun, 4 Feb 2001 04:57:24 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f149v9609528
for ; Sun, 4 Feb 2001 04:57:10 -0500 (EST)
Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23])
by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id KAA33530
for ; Sun, 4 Feb 2001 10:56:59 +0100
From: julian_satran@il.ibm.com
Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237])
by d12relay02.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id KAA51944
for ; Sun, 4 Feb 2001 10:56:55 +0100
Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569E9.00369C79 ; Sun, 4 Feb 2001 10:56:30 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: ips@ece.cmu.edu
Message-ID:
Date: Sun, 4 Feb 2001 11:52:00 +0200
Subject: Re: Section 4.1 (Login Phase)
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
wrong. Julo
Santosh Rao on 02/02/2001 18:46:20
Please respond to Santosh Rao
To: Julian Satran/Haifa/IBM@IBMIL
cc: santoshr@hpcuhe.cup.hp.com (Santosh Rao)
Subject: Section 4.1 (Login Phase)
Julian,
The following sentence :
"Login reject with Final bit 0 is a format error."
needs to be removed from the draft. I believe the
Reject PDU is intended to communicate all format
errors on any PDU (?)
Regards,
Santosh
--
#################################
Santosh Rao
Software Design Engineer,
HP, Cupertino.
email : santoshr@cup.hp.com
Phone : 408-447-3751
#################################
From owner-ips@ece.cmu.edu Sun Feb 4 16:53:33 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA11997
for ; Sun, 4 Feb 2001 16:53:32 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f14K1Wf23040
for ips-outgoing; Sun, 4 Feb 2001 15:01:32 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from hotmail.com (f103.law4.hotmail.com [216.33.149.103])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f14K0j623022
for ; Sun, 4 Feb 2001 15:00:45 -0500 (EST)
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
Sun, 4 Feb 2001 12:00:39 -0800
Received: from 203.124.2.57 by lw4fd.law4.hotmail.msn.com with HTTP; Sun, 04 Feb 2001 20:00:39 GMT
X-Originating-IP: [203.124.2.57]
From: "Santosh Rao"
To: ips@ece.cmu.edu
Subject: Re: Section 4.1 (Login Phase)
Date: Sun, 04 Feb 2001 20:00:39
Mime-Version: 1.0
Content-Type: text/html
Message-ID:
X-OriginalArrivalTime: 04 Feb 2001 20:00:39.0584 (UTC) FILETIME=[25996A00:01C08EE5]
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Julian,
Are you saying the Reject is NOT meant to be used for all format errors (?)
How then is the following description of Reject PDU (Section 2.19) to be interpreted :
"It may happen that a target receives a message with a format error
(inconsistent fields, reserved fields not 0, inexistent LUN etc.) or
a digest error (invalid payload or header). The target returns the
header of the message in error as the data of the response.
2.20 Reason
The reject Reason is coded as follows:
1 - Format Error
2 - Header Digest Error
3 - Payload Digest Error ""
Why are 2 mechanisms required to indicate format errors ? The one in Login response (reject with F=0) seems redundant.
- Santosh
>From: julian_satran@il.ibm.com
>To: ips@ece.cmu.edu
>Subject: Re: Section 4.1 (Login Phase)
>Date: Sun, 4 Feb 2001 11:52:00 +0200
>
>
>
>wrong. Julo
>
>Santosh Rao on 02/02/2001 18:46:20
>
>Please respond to Santosh Rao
>
>To: Julian Satran/Haifa/IBM@IBMIL
>cc: santoshr@hpcuhe.cup.hp.com (Santosh Rao)
>Subject: Section 4.1 (Login Phase)
>
>
>
>
>Julian,
>
>The following sentence :
>
>"Login reject with Final bit 0 is a format error."
>
>needs to be removed from the draft. I believe the
>Reject PDU is intended to communicate all format
>errors on any PDU (?)
>
>Regards,
>Santosh
>
>--
>#################################
>Santosh Rao
>Software Design Engineer,
>HP, Cupertino.
>email : santoshr@cup.hp.com
>Phone : 408-447-3751
>#################################
>
>
>
Get your FREE download of MSN Explorer at http://explorer.msn.com
From owner-ips@ece.cmu.edu Sun Feb 4 16:58:23 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA12030
for ; Sun, 4 Feb 2001 16:58:22 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f14KdWV24131
for ips-outgoing; Sun, 4 Feb 2001 15:39:32 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from hotmail.com (f79.law4.hotmail.com [216.33.149.79])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f14Kcf624095
for ; Sun, 4 Feb 2001 15:38:42 -0500 (EST)
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
Sun, 4 Feb 2001 12:38:36 -0800
Received: from 203.124.2.57 by lw4fd.law4.hotmail.msn.com with HTTP; Sun, 04 Feb 2001 20:38:35 GMT
X-Originating-IP: [203.124.2.57]
From: "Santosh Rao"
To: ips@ece.cmu.edu
Subject: iSCSI : Abort Task "connection allegiance"
Date: Sun, 04 Feb 2001 20:38:35 -0000
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID:
X-OriginalArrivalTime: 04 Feb 2001 20:38:36.0036 (UTC) FILETIME=[72786C40:01C08EEA]
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
>To: ips@ece.cmu.edu Subject: iSCSI-related conclusions from Orlando interim
>Meeting From: Black_David@emc.com Date: Fri, 19 Jan 2001 19:29:36 -0500
>(2) Error Recovery
>- Abort WARNING will be added to the draft.
>- Immediate Delivery of Aborts and similar task management commands
> may have unexpected results when multiple TCP connections are in use
> because Abort, Clear Task Set, etc. may bypass command(s) to be
> aborted/cleared on other TCP connections.
>- Ordered Delivery should be used instead when this is a concern.
-------------------------------------------------------------------
Julian & All,
For the above issue, the Abort Task I/O Error Recovery is udertaken
when an I/O times out (O.S. specific SCSI ULP timer has expired). Under
such circumstances, it is preferrable to have an expedited error
recovery which is not hindered by stalls in its processing due to the
target waiting for missing CmdSNs.
[which are applicable when the Abort Task is sent with a non-0 CmdSN].
In the interests of keeping I/O timeout recovery quick, the
"connection allegiance" policy, currently applicable to all phases of a
command, should be extended to the Abort Task for that command as well
and the Abort Task be sent with a CmdSN of 0.
Regards,
Santosh
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com
From owner-ips@ece.cmu.edu Mon Feb 5 02:50:55 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id CAA02108
for ; Mon, 5 Feb 2001 02:50:54 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f155obm09362
for ips-outgoing; Mon, 5 Feb 2001 00:50:37 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f155o1609326
for ; Mon, 5 Feb 2001 00:50:01 -0500 (EST)
Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22])
by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id GAA70076
for ; Mon, 5 Feb 2001 06:49:54 +0100
From: julian_satran@il.ibm.com
Received: from d12mta02.de.ibm.com (d12mta01_cs0 [9.165.222.237])
by d12relay01.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id GAA233362
for ; Mon, 5 Feb 2001 06:49:50 +0100
Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569EA.002006FC ; Mon, 5 Feb 2001 06:49:49 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: ips@ece.cmu.edu
Message-ID:
Date: Mon, 5 Feb 2001 07:45:16 +0200
Subject: Re: Section 4.1 (Login Phase)
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Santosh,
Norton Antivirus reports a virus in your attachment.
You may want to send plain text.
Julo
"Santosh Rao" on 04/02/2001 22:00:39
Please respond to "Santosh Rao"
To: ips@ece.cmu.edu
cc:
Subject: Re: Section 4.1 (Login Phase)
- att1.htm
From owner-ips@ece.cmu.edu Mon Feb 5 12:23:31 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id MAA12330
for ; Mon, 5 Feb 2001 12:23:31 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f15FCkd01613
for ips-outgoing; Mon, 5 Feb 2001 10:12:46 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from chmls06.mediaone.net (chmls06.mediaone.net [24.147.1.144])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f15FCK601597
for ; Mon, 5 Feb 2001 10:12:20 -0500 (EST)
Received: from breinhold ([65.192.191.129])
by chmls06.mediaone.net (8.11.1/8.11.1) with SMTP id f15FC6K00140;
Mon, 5 Feb 2001 10:12:06 -0500 (EST)
From: "Barry Reinhold"
To: ,
Subject: RE: ISCSI: Immediate data extending beyond a single PDU
Date: Mon, 5 Feb 2001 10:11:36 -0500
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6700
Importance: Normal
In-Reply-To:
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Julian,
I think one could carry on a debate about what the draft currently states
about the coupling of immediate and unsolicited data - but it is probably
more productive to figure out if we want to have both the concept of
immediate data and unsolicited data and then worry about the wording in the
draft.
I am of the opinion that immediate data (data within a single PDU) is a
useful distincition from unsolicited data (iSCSI DATA PDU sent without R2T).
The reasoning behind this is that there are a number of write IOs that can
be satisfied by providing a buffer of 8K or less. If this data is contained
in the command PDU the context can be created for the command, and then the
data just happens to be sitting there with the context to process it. Thus
the target can send the iSCSI response PDU without creating any special
context or mechansims - should have nice low latency characteristics for
small writes.
For devices that don't want to use the initiator task tag for establishing
context of an iSCSI DATA PDU, immediate data is a nice thing. An iSCSI DATA
PDU without a target task tag may not be such a nice thing.
Question here - What would you lose if you supported only immediate data?
It seems like this meets some of the common application requirements that I
am aware of and is friendly to what I think is the more common approach to
target processing of DATA PDUs.
>-----Original Message-----
>From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
>julian_satran@il.ibm.com
>Sent: Sunday, February 04, 2001 5:20 AM
>To: ips@ece.cmu.edu
>Subject: RE: ISCSI: Immediate data extending beyond a single PDU
>
>
>
>
>Barry,
>
>I think that the current draft does not allow you to accept immediate but
>not unsolicited.
>You can either allow unsolicited (including immediate) or request always
>R2T.
>
>And yes - if the expected length is larger that what you sent as
>unsolicited (even if the unsolicited was less than the limit) the target is
>supposed to send R2T (there is only one unsolicited burst per command).
>
>The limit for immediate data is implicit - the max-PDU length and is
>less-then-or-equal the first burst.
>
>Your proposal is interesting but I doubt that it is very useful.
>
>Unsolicited data help avoid an RTT on writes but require resources to be
>allocated (that is why a target may not accept them).
>
>Immediate data is an "optimized" form of unsolicited data in which the
>command and data are collapsed to one unit. Nevertheless the immediate data
>are unsolicited and require resources to be allocated ahead of time. If you
>have larger amounts to send than what fits into one PDU the "collapse gain"
>is minimal and does not justify the added complexity of 2 options.
>
>Regards,
>Julo
>
>
>
>"Barry Reinhold" on 02/02/2001 21:06:29
>
>Please respond to "Barry Reinhold"
>
>To: Julian Satran/Haifa/IBM@IBMIL
>cc:
>Subject: RE: ISCSI: Immediate data extending beyond a single PDU
>
>
>
>
>Julian,
> I agree with the conceptual idea of immediate data, but further
>refinement
>is required. As a receiver, if I have allowed immediate data, but not
>unsolicited data, and I get a write command with an expected data transfer
>length that is greater then the amount of data in the command PDU, do I
>send
>an R2T or not (or from the sender's point of view does he wait for an R2T
>to
>complete the data transfer?) The draft is not clear on this, yet it is an
>externally visible behavior that needs to be defined for interoperability.
>
>I believe that what we need here is a refinement of two distinct concepts,
>that of immediate data and that of unsolicited data. Here is my take at it:
>
>1. Immediate data - Data that is sent within a single iSCSI SCSI command
>PDU. Immediate data shall be fully contained within a single iSCSI command
>PDU, and shall be limitted by the negotiated value of ImmediateData. The
>size of Immediate data is not subject to the limits of FirstBurst.
>
>2. Unsolicited data - Data that may be transfered to a target without an
>explicit R2T being sent to request its transfer. Unsolicited data is sent
>in
>iSCSI SCSI data PDUs in which the target transfer tag is invalid. The
>maximum amount of unsolicited data is negotiated between the initiator and
>target through the FirstBurst key. If unsolicited data is supported by the
>target, upon receipt of a write command, the target shall not send an R2T
>until an iSCSI Data PDU with the Final bit set has been received. After
>reception of an iSCSI Data PDU with the Final bit set R2Ts shall be sent by
>the target to request transmission of any additional data associated with
>this command. The initiator shall not send any additional unsolicited data
>for this command.
>
>Interaction between Immediate data and Unsolicited data -
>
>Immediate data and Unsolicited data are features that may be selected
>independently of one another. Support of either is optional.
>
>Note: Based on the current specification of FirstBurst, support for
>unsolicited data is not optional. One must select between a range of
>1-65536. I think we would want to make this optional and indicate non
>support for unsolicited data by setting FirstBurst to 0.
>
>
>>-----Original Message-----
>>From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
>>julian_satran@il.ibm.com
>>Sent: Friday, February 02, 2001 6:41 AM
>>To: ips@ece.cmu.edu
>>Subject: Re: ISCSI: Immediate data extending beyond a single PDU
>>
>>
>>
>>
>>Barry,
>>
>>There is a misunderstanding:
>>
>>You can send EITHER immediate data OR a sequence of unsolicited PDUs.
>>
>>Immediate data was conceived for short writes - for which an initiator
>does
>>not want to do
>>two socket operations or 2 HBA operations.
>>
>>Larger unsolicited bursts can be sent with a sequence and then we felt
>that
>>you won't gain too much by having the first one glued to the command.
>>
>>
>>Julo
>>
>>"Barry Reinhold - Lamprey" on 01/02/2001
>22:04:28
>>
>>Please respond to "Barry Reinhold - Lamprey"
>>
>>To: Julian Satran/Haifa/IBM@IBMIL
>>cc: "ISCSI"
>>Subject: ISCSI: Immediate data extending beyond a single PDU
>>
>>
>>
>>
>>Julian,
>>
>>My understanding of your email in conjunction with the current draft is:
>>
>>The initiator may send zero or more immediate data bytes in the CMD PDU,
>>and
>>may send additional unsolicited DATA PDUs with further immediate data,
>with
>>the
>>last PDU having the F bit set. The total amount of immediate data to be
>>limited
>>by the value negotiated at login.
>>
>>There are two issues I have with this:
>>
>>First, the target does not know how much immediate data has being sent.
>>There
>>is no indication in the CMD frame to indicate that more immediate data is
>>to
>>follow (whether or not the CMD frame contains any immediate data). The
>>initiator is free to send any amount, from zero up to the negotiated
>value.
>>Therefore, the target will likely send an R2T to the host at whatever
>>boundry
>>it believes is present - which may cause a retransmission of much or all
>of
>>the
>>immediate data.
>>
>>Second, the additional unsolicited PDU's do not contain a target
>>task/transfer
>>tag. As such, in the mainline processing, you are perhaps forcing the
>>target
>>to
>>look up the Initiator Task Tag in its list of active i/o's. This may exact
>>a
>>heavy performance penalty as optimizing this lookup maybe
>cost-prohibitive.
>>The
>>only other scenarios in which the target had to perform this lookup was in
>>low-occurance error/abort conditions.
>>
>>Are there negative implications to keeping immediate data to a single PDU?
>>
>>-- Barry
>>
>>
>>julian_satran@il.ibm.com wrote:
>>
>>> Barry,
>>>
>>> Immediate data is "attached to the command". If your unsolicited data
>>(the
>>> so called initial burst) is larger that what you are ready to commit to
>a
>>> single PDU you can send several PDUs with the last having the F bit.
>>After
>>> that the target can decide that it wants more and send an R2T or send
>>> status.
>>>
>>> You are no supposed to send both immediate and separate PDU.
>>>
>>> I admit that the text is bit confusing (in 1.2.5) and there is an error
>>> also in the appendix.
>>> 2.7.1 talk about unsolicited data (not immediate) (the context is data
>>> PDUs).
>>>
>>> The new version of 1.2.5 now reads:
>>>
>>> Unsolicited data can be sent as part of an iSCSI command PDU
>>("immediate
>>> data") or an in separate iSCSI data PDUs. An initiator may send
>>> unsolicited data either as immediate (up to the negotiated maximum
>PDU
>>> size - DataPDULength - disconnect-reconnect mode page) or in a
>>separate
>>> PDU sequence (up to the negotiated limit - FirstBurstSize -
>>> disconnect-reconnect mode page). All subsequent data have to be
>>> solicited. The maximum size of an individual data PDU or the
>>> immediate-part of the initial unsolicited burst as well as the
>initial
>>> burst size MAY be negotiated at login.
>>>
>>> Thanks,
>>> Julo
>>>
>>> "Barry Reinhold - Lamprey" on 29/01/2001
>>16:34:30
>>>
>>> Please respond to "Barry Reinhold - Lamprey"
>>>
>>> To: Julian Satran/Haifa/IBM@IBMIL
>>> cc: "Jon Sreekanth" , "James Smart"
>>>
>>> Subject:
>>>
>>> Issue:
>>>
>>> I am unclear on how to respond to an iSCSI SCSI command PDU when there
>is
>>> immediate data but not enough to meet the requirements of the IO
>>operation
>>> as given in the expected data transfer length field. Should an R2T be
>>> issued
>>> or not?
>>>
>>> Background:
>>>
>>> Clause 2.7.1 in 03 reads:
>>>
>>> "2.7.1 F (Final) bit
>>>
>>> This bit is 1 for the last PDU of immediate data or the last PDU of a
>>> sequence answering a R2T."
>>>
>>> This suggests that immediate data can extend beyond the iSCSI SCSI CMD
>>PDU
>>> (there is no final bit on the command PDU)but the closet thing we have
>to
>>a
>>> definition for immediate data, which is in clause 1.2.5 below, suggests
>>> that
>>> immediate data is limitted to the command PDU:
>>>
>>> "Outgoing SCSI data (initiator to target - user data or command
>>> parameters) will be sent as either solicited data or unsolicited
>>> data. Solicited data are sent in response to Ready To Transfer (R2T)
>>> PDUs. Unsolicited data can be part of an iSCSI command PDU
>>> ("immediate data") or an iSCSI data PDU. An initiator may send
>>> unsolicited data (immediate or in a separate PDU) up to the
>>> negotiated limit (initial burst size - mode page 02h). All subsequent
>>> data have to be solicited. The maximum size of an individual data
>>> PDU or the immediate-part of the initial unsolicited burst MAY be
>>> negotiated at login (maximum connect size - mode page 02h)."
>>>
>>> Unsolicited data is bounded in a number of ways, but most significant to
>>> this issue is in the description of the login key useR2t which states:
>>>
>>> "Note than only the first
>>> outgoing data item (either immediate data or a separate PDU) can be
>>> sent unsolicited by a R2T."
>>>
>>> Given the above it is unclear to me if the concept of immediate data is:
>>>
>>> 1. Write data that can be sent without an R2T (unsolicited write
>data)and
>>> starts in the command PDU.
>>> 2. Write data that is entirely contained within the command PDU. (my
>>> initial
>>> concept of immediate data)
>>> 3. The non-zero portion of data, in a possible multi-PDU write
>operation,
>>> that is contained in the iSCSI SCSI command PDU. Where the remaining
>PDUs
>>> of
>>> the write operation must be solicited with an R2T.
>>>
>>> Given the current definition of the Final bit and the current limits on
>>> unsolicited data it is unclear to me what a target should do when
>>receiving
>>> an iSCSI SCSI command PDU that has immediate data in it, but not
>>sufficient
>>> data to meet the expected data transfer length specificied in the
>>command.
>>> Does an R2T go out of not?
>>>
>>> Barry Reinhold
>>> 603-659-0885
>>
>>Barry Reinhold
>>603-659-0885
>>
>>
>>
>>
>
>
>
>
From owner-ips@ece.cmu.edu Mon Feb 5 14:40:12 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA15666
for ; Mon, 5 Feb 2001 14:40:11 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f15HMxY07232
for ips-outgoing; Mon, 5 Feb 2001 12:22:59 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f15HJi607067
for ; Mon, 5 Feb 2001 12:19:44 -0500 (EST)
Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23])
by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id SAA59890
for ; Mon, 5 Feb 2001 18:19:35 +0100
From: julian_satran@il.ibm.com
Received: from d12mta02.de.ibm.com (d12mta02_cs0 [9.165.222.253])
by d12relay02.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id SAA76960
for ; Mon, 5 Feb 2001 18:19:33 +0100
Received: by d12mta02.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569EA.005F2869 ; Mon, 5 Feb 2001 18:19:22 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: ips@ece.cmu.edu
Message-ID:
Date: Mon, 5 Feb 2001 18:50:17 +0200
Subject: RE: ISCSI: Immediate data extending beyond a single PDU
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Barry,
You are suggesting support of immediate data as separate from the support
for unsolicited data as a target option?
That could be accomodated by stating that support for immediate data is
not implied by R2T NO.
If there are no strong objections on the list I am ready to remove/correct
the offending statement in the appendix.
However we will maintain only one initial burst per command (either
immediate or in several separate PDUs).
Regards,
Julo
"Barry Reinhold" on 05/02/2001 17:11:36
Please respond to "Barry Reinhold"
To: Julian Satran/Haifa/IBM@IBMIL, ips@ece.cmu.edu
cc:
Subject: RE: ISCSI: Immediate data extending beyond a single PDU
Julian,
I think one could carry on a debate about what the draft currently
states
about the coupling of immediate and unsolicited data - but it is probably
more productive to figure out if we want to have both the concept of
immediate data and unsolicited data and then worry about the wording in the
draft.
I am of the opinion that immediate data (data within a single PDU) is
a
useful distincition from unsolicited data (iSCSI DATA PDU sent without
R2T).
The reasoning behind this is that there are a number of write IOs that can
be satisfied by providing a buffer of 8K or less. If this data is contained
in the command PDU the context can be created for the command, and then the
data just happens to be sitting there with the context to process it. Thus
the target can send the iSCSI response PDU without creating any special
context or mechansims - should have nice low latency characteristics for
small writes.
For devices that don't want to use the initiator task tag for
establishing
context of an iSCSI DATA PDU, immediate data is a nice thing. An iSCSI DATA
PDU without a target task tag may not be such a nice thing.
Question here - What would you lose if you supported only immediate
data?
It seems like this meets some of the common application requirements that I
am aware of and is friendly to what I think is the more common approach to
target processing of DATA PDUs.
>-----Original Message-----
>From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
>julian_satran@il.ibm.com
>Sent: Sunday, February 04, 2001 5:20 AM
>To: ips@ece.cmu.edu
>Subject: RE: ISCSI: Immediate data extending beyond a single PDU
>
>
>
>
>Barry,
>
>I think that the current draft does not allow you to accept immediate but
>not unsolicited.
>You can either allow unsolicited (including immediate) or request always
>R2T.
>
>And yes - if the expected length is larger that what you sent as
>unsolicited (even if the unsolicited was less than the limit) the target
is
>supposed to send R2T (there is only one unsolicited burst per command).
>
>The limit for immediate data is implicit - the max-PDU length and is
>less-then-or-equal the first burst.
>
>Your proposal is interesting but I doubt that it is very useful.
>
>Unsolicited data help avoid an RTT on writes but require resources to be
>allocated (that is why a target may not accept them).
>
>Immediate data is an "optimized" form of unsolicited data in which the
>command and data are collapsed to one unit. Nevertheless the immediate
data
>are unsolicited and require resources to be allocated ahead of time. If
you
>have larger amounts to send than what fits into one PDU the "collapse
gain"
>is minimal and does not justify the added complexity of 2 options.
>
>Regards,
>Julo
>
>
>
>"Barry Reinhold" on 02/02/2001 21:06:29
>
>Please respond to "Barry Reinhold"
>
>To: Julian Satran/Haifa/IBM@IBMIL
>cc:
>Subject: RE: ISCSI: Immediate data extending beyond a single PDU
>
>
>
>
>Julian,
> I agree with the conceptual idea of immediate data, but further
>refinement
>is required. As a receiver, if I have allowed immediate data, but not
>unsolicited data, and I get a write command with an expected data transfer
>length that is greater then the amount of data in the command PDU, do I
>send
>an R2T or not (or from the sender's point of view does he wait for an R2T
>to
>complete the data transfer?) The draft is not clear on this, yet it is an
>externally visible behavior that needs to be defined for interoperability.
>
>I believe that what we need here is a refinement of two distinct concepts,
>that of immediate data and that of unsolicited data. Here is my take at
it:
>
>1. Immediate data - Data that is sent within a single iSCSI SCSI command
>PDU. Immediate data shall be fully contained within a single iSCSI command
>PDU, and shall be limitted by the negotiated value of ImmediateData. The
>size of Immediate data is not subject to the limits of FirstBurst.
>
>2. Unsolicited data - Data that may be transfered to a target without an
>explicit R2T being sent to request its transfer. Unsolicited data is sent
>in
>iSCSI SCSI data PDUs in which the target transfer tag is invalid. The
>maximum amount of unsolicited data is negotiated between the initiator and
>target through the FirstBurst key. If unsolicited data is supported by the
>target, upon receipt of a write command, the target shall not send an R2T
>until an iSCSI Data PDU with the Final bit set has been received. After
>reception of an iSCSI Data PDU with the Final bit set R2Ts shall be sent
by
>the target to request transmission of any additional data associated with
>this command. The initiator shall not send any additional unsolicited data
>for this command.
>
>Interaction between Immediate data and Unsolicited data -
>
>Immediate data and Unsolicited data are features that may be selected
>independently of one another. Support of either is optional.
>
>Note: Based on the current specification of FirstBurst, support for
>unsolicited data is not optional. One must select between a range of
>1-65536. I think we would want to make this optional and indicate non
>support for unsolicited data by setting FirstBurst to 0.
>
>
>>-----Original Message-----
>>From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
>>julian_satran@il.ibm.com
>>Sent: Friday, February 02, 2001 6:41 AM
>>To: ips@ece.cmu.edu
>>Subject: Re: ISCSI: Immediate data extending beyond a single PDU
>>
>>
>>
>>
>>Barry,
>>
>>There is a misunderstanding:
>>
>>You can send EITHER immediate data OR a sequence of unsolicited PDUs.
>>
>>Immediate data was conceived for short writes - for which an initiator
>does
>>not want to do
>>two socket operations or 2 HBA operations.
>>
>>Larger unsolicited bursts can be sent with a sequence and then we felt
>that
>>you won't gain too much by having the first one glued to the command.
>>
>>
>>Julo
>>
>>"Barry Reinhold - Lamprey" on 01/02/2001
>22:04:28
>>
>>Please respond to "Barry Reinhold - Lamprey"
>>
>>To: Julian Satran/Haifa/IBM@IBMIL
>>cc: "ISCSI"
>>Subject: ISCSI: Immediate data extending beyond a single PDU
>>
>>
>>
>>
>>Julian,
>>
>>My understanding of your email in conjunction with the current draft is:
>>
>>The initiator may send zero or more immediate data bytes in the CMD PDU,
>>and
>>may send additional unsolicited DATA PDUs with further immediate data,
>with
>>the
>>last PDU having the F bit set. The total amount of immediate data to be
>>limited
>>by the value negotiated at login.
>>
>>There are two issues I have with this:
>>
>>First, the target does not know how much immediate data has being sent.
>>There
>>is no indication in the CMD frame to indicate that more immediate data is
>>to
>>follow (whether or not the CMD frame contains any immediate data). The
>>initiator is free to send any amount, from zero up to the negotiated
>value.
>>Therefore, the target will likely send an R2T to the host at whatever
>>boundry
>>it believes is present - which may cause a retransmission of much or all
>of
>>the
>>immediate data.
>>
>>Second, the additional unsolicited PDU's do not contain a target
>>task/transfer
>>tag. As such, in the mainline processing, you are perhaps forcing the
>>target
>>to
>>look up the Initiator Task Tag in its list of active i/o's. This may
exact
>>a
>>heavy performance penalty as optimizing this lookup maybe
>cost-prohibitive.
>>The
>>only other scenarios in which the target had to perform this lookup was
in
>>low-occurance error/abort conditions.
>>
>>Are there negative implications to keeping immediate data to a single
PDU?
>>
>>-- Barry
>>
>>
>>julian_satran@il.ibm.com wrote:
>>
>>> Barry,
>>>
>>> Immediate data is "attached to the command". If your unsolicited data
>>(the
>>> so called initial burst) is larger that what you are ready to commit to
>a
>>> single PDU you can send several PDUs with the last having the F bit.
>>After
>>> that the target can decide that it wants more and send an R2T or send
>>> status.
>>>
>>> You are no supposed to send both immediate and separate PDU.
>>>
>>> I admit that the text is bit confusing (in 1.2.5) and there is an
error
>>> also in the appendix.
>>> 2.7.1 talk about unsolicited data (not immediate) (the context is data
>>> PDUs).
>>>
>>> The new version of 1.2.5 now reads:
>>>
>>> Unsolicited data can be sent as part of an iSCSI command PDU
>>("immediate
>>> data") or an in separate iSCSI data PDUs. An initiator may send
>>> unsolicited data either as immediate (up to the negotiated maximum
>PDU
>>> size - DataPDULength - disconnect-reconnect mode page) or in a
>>separate
>>> PDU sequence (up to the negotiated limit - FirstBurstSize -
>>> disconnect-reconnect mode page). All subsequent data have to be
>>> solicited. The maximum size of an individual data PDU or the
>>> immediate-part of the initial unsolicited burst as well as the
>initial
>>> burst size MAY be negotiated at login.
>>>
>>> Thanks,
>>> Julo
>>>
>>> "Barry Reinhold - Lamprey" on 29/01/2001
>>16:34:30
>>>
>>> Please respond to "Barry Reinhold - Lamprey"
>>>
>>> To: Julian Satran/Haifa/IBM@IBMIL
>>> cc: "Jon Sreekanth" , "James Smart"
>>>
>>> Subject:
>>>
>>> Issue:
>>>
>>> I am unclear on how to respond to an iSCSI SCSI command PDU when there
>is
>>> immediate data but not enough to meet the requirements of the IO
>>operation
>>> as given in the expected data transfer length field. Should an R2T be
>>> issued
>>> or not?
>>>
>>> Background:
>>>
>>> Clause 2.7.1 in 03 reads:
>>>
>>> "2.7.1 F (Final) bit
>>>
>>> This bit is 1 for the last PDU of immediate data or the last PDU of
a
>>> sequence answering a R2T."
>>>
>>> This suggests that immediate data can extend beyond the iSCSI SCSI CMD
>>PDU
>>> (there is no final bit on the command PDU)but the closet thing we have
>to
>>a
>>> definition for immediate data, which is in clause 1.2.5 below, suggests
>>> that
>>> immediate data is limitted to the command PDU:
>>>
>>> "Outgoing SCSI data (initiator to target - user data or command
>>> parameters) will be sent as either solicited data or unsolicited
>>> data. Solicited data are sent in response to Ready To Transfer
(R2T)
>>> PDUs. Unsolicited data can be part of an iSCSI command PDU
>>> ("immediate data") or an iSCSI data PDU. An initiator may send
>>> unsolicited data (immediate or in a separate PDU) up to the
>>> negotiated limit (initial burst size - mode page 02h). All
subsequent
>>> data have to be solicited. The maximum size of an individual data
>>> PDU or the immediate-part of the initial unsolicited burst MAY be
>>> negotiated at login (maximum connect size - mode page 02h)."
>>>
>>> Unsolicited data is bounded in a number of ways, but most significant
to
>>> this issue is in the description of the login key useR2t which states:
>>>
>>> "Note than only the first
>>> outgoing data item (either immediate data or a separate PDU) can be
>>> sent unsolicited by a R2T."
>>>
>>> Given the above it is unclear to me if the concept of immediate data
is:
>>>
>>> 1. Write data that can be sent without an R2T (unsolicited write
>data)and
>>> starts in the command PDU.
>>> 2. Write data that is entirely contained within the command PDU. (my
>>> initial
>>> concept of immediate data)
>>> 3. The non-zero portion of data, in a possible multi-PDU write
>operation,
>>> that is contained in the iSCSI SCSI command PDU. Where the remaining
>PDUs
>>> of
>>> the write operation must be solicited with an R2T.
>>>
>>> Given the current definition of the Final bit and the current limits on
>>> unsolicited data it is unclear to me what a target should do when
>>receiving
>>> an iSCSI SCSI command PDU that has immediate data in it, but not
>>sufficient
>>> data to meet the expected data transfer length specificied in the
>>command.
>>> Does an R2T go out of not?
>>>
>>> Barry Reinhold
>>> 603-659-0885
>>
>>Barry Reinhold
>>603-659-0885
>>
>>
>>
>>
>
>
>
>
From owner-ips@ece.cmu.edu Mon Feb 5 14:41:39 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA15727
for ; Mon, 5 Feb 2001 14:41:36 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f15Hbnx07902
for ips-outgoing; Mon, 5 Feb 2001 12:37:49 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from main.connectcom.net ([204.247.22.3])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f15Hba607890
for ; Mon, 5 Feb 2001 12:37:36 -0500 (EST)
Received: from yp_portable (anubis.advansys.com [204.247.22.2]) by main.connectcom.net with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id 11CPQF1Q; Mon, 5 Feb 2001 09:36:05 -0800
From: "Y P Cheng"
To: , "Santosh Rao"
Cc: "Julian_Satran@Il. Ibm. Com"
Subject: RE: iSCSI : Abort Task "connection allegiance"
Date: Mon, 5 Feb 2001 09:33:55 -0800
Message-ID: <000101c08f99$d084b460$90c809c0@yp_portable.advansys.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
X-MIMEOLE: Produced By Microsoft MimeOLE V4.72.3110.3
Importance: Normal
In-Reply-To:
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Santosh wrote:
> For the above issue, the Abort Task I/O Error Recovery is udertaken
> when an I/O times out (O.S. specific SCSI ULP timer has expired). Under
> such circumstances, it is preferrable to have an expedited error
> recovery which is not hindered by stalls in its processing due to the
> target waiting for missing CmdSNs.
> [which are applicable when the Abort Task is sent with a non-0 CmdSN].
There are two reasons for issuing abort: 1) application software exits and
OS needs cleaning up, and 2) a request times out and it needs clean restart.
In either cases, we don't want the abort arrives there before the one to be
aborted. Hence, using zero CmdSN is unwise. If the abort gets there first,
the target will complete the abort quickly and later executes the one to be
aborted anyway.
> In the interests of keeping I/O timeout recovery quick, the
> "connection allegiance" policy, currently applicable to all phases of a
> command, should be extended to the Abort Task for that command as well
> and the Abort Task be sent with a CmdSN of 0.
I don't follow this logic at all. Are you trying to say that if the target
gets an abort, it should enter connection allegiance until the one to be
aborted arrives? This would real make a mess on the target microcode. How
do we even know the aborted request would arrive? IMHO, abort should never
have a zero CmdSN.
Y.P.
From owner-ips@ece.cmu.edu Mon Feb 5 16:31:13 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id QAA18138
for ; Mon, 5 Feb 2001 16:31:12 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f15JKvr15292
for ips-outgoing; Mon, 5 Feb 2001 14:20:57 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f15JJm615226
for ; Mon, 5 Feb 2001 14:19:48 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f15KTe073245;
Mon, 5 Feb 2001 12:29:40 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: "Y P Cheng" ,
"David Robinson" ,
Subject: RE: iSCSI Data Integrity - Digests
Date: Mon, 5 Feb 2001 11:18:09 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
In-Reply-To: <003801c08d74$d3189420$90c809c0@yp_portable.advansys.com>
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> Hi, Doug:
>
> The Charters of this WG prohibits us discussing any change of TCP protocol
> and I was not talking about changing any TCP protocol. I am
> quite familiar
> with the history why urgent pointer wouldn't work as so many TCP
> gurus were
> too quickly to point out. This WG does not recommend the use of urgent
> pointer nor does it endorse the placement of a command/status PDU in a
> separate TCP segment.
>
> However, for microcode that running in my iSCSI interface chip with my TCP
> acceleration implementation, I can place each command/status PDU in a
> separate TCP segment without violating the TCP protocol.
> Similarly, in the
> manner I understood about TCP if I turn on the urgent pointer when I send
> the PDU to another iSCSI chip of which I came to know at login, I don't
> believe I have altered TCP.
Y.P.
You are free to add non-standard TCP and perhaps both server and client may
implement the same level of alteration. Change aggressiveness of the TCP
congestion algorithm, force framing, implement non-standard definitions of
urgent pointers, add special flags, and seldom seen options. View TCP as
important in name only or wire only. An alternative to chaos generated by
multitudes of such clever creations would be adoption of SCTP to provide
options already found within this standard.
iSCSI is a good example why transports are not easily developed. If to
allow fail-over with iSCSI, even StatSN should be unique across all
connections and a strict monotonic sequence should be mandated for both
CmdSN or StatSN within every PDU (rename to ClientSN and ServerSN.) Better
yet, adopt the SCTP structures within TCP (replace the SCTP ports with a
pseudo-frame length and allow padding to fixed intervals for frame
recovery.) I fail to see how either client or server knows the state of the
other and how to prevent a cascade, and what benefit is derived by the
complex process of rediscovery of discarded relationships.
Doug
> Some people in this WG might not like what could be done by each iSCSI
> interface chip designer. However, this method of doing something
> special in
> a SCSI device not available by others gives one a unique
> advantage and is a
> common practice by the SCSI industry for many years. I will not be
> surprised that each iSCSI interface chip will have its own bag of
> tricks to
> accelerate its performance and still totally confirm the iSCSI
> specifications. This is how the technology world is. :-)
>
> Y.P.
>
>
From owner-ips@ece.cmu.edu Mon Feb 5 17:56:09 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA19968
for ; Mon, 5 Feb 2001 17:56:08 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f15LVoo21391
for ips-outgoing; Mon, 5 Feb 2001 16:31:50 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from main.connectcom.net (ns1.advansys.com [204.247.22.3] (may be forged))
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f15LVe621378
for ; Mon, 5 Feb 2001 16:31:41 -0500 (EST)
Received: from yp_portable (anubis.advansys.com [204.247.22.2]) by main.connectcom.net with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id 11CPQFRW; Mon, 5 Feb 2001 13:31:38 -0800
From: "Y P Cheng"
To: "Douglas Otis" ,
"David Robinson" ,
Subject: RE: iSCSI Data Integrity - Digests
Date: Mon, 5 Feb 2001 13:29:25 -0800
Message-ID: <001701c08fba$b6cc3860$90c809c0@yp_portable.advansys.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
X-MIMEOLE: Produced By Microsoft MimeOLE V4.72.3110.3
Importance: Normal
In-Reply-To:
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> You are free to add non-standard TCP and perhaps both server and
> client may
> implement the same level of alteration. Change aggressiveness of the TCP
> congestion algorithm, force framing, implement non-standard definitions of
> urgent pointers, add special flags, and seldom seen options. View TCP as
> important in name only or wire only. An alternative to chaos generated by
> multitudes of such clever creations would be adoption of SCTP to provide
> options already found within this standard.
Doug,
May be I am mistaken, but I don't think what I am suggesting is doing
non-standard TCP. What is a non-standard TCP if I follow all the RFCs
approved by IETF? What I think you are referring to the non-standard
implementation, i.e., not working working with existing TCP stacks.
The urgent pointer, 32-bit TCP window, one segment per PDU, even with the
addition of SACK are all part of "standard" TCP, just not every TCP
implementations are supporting them. This is why option negotiation comes
in. I hope I am not mistaken.
Y.P.
From owner-ips@ece.cmu.edu Mon Feb 5 17:56:48 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA19983
for ; Mon, 5 Feb 2001 17:56:47 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f15Krn919650
for ips-outgoing; Mon, 5 Feb 2001 15:53:49 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [168.159.208.52])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f15Krg619640
for ; Mon, 5 Feb 2001 15:53:42 -0500 (EST)
Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2650.21)
id <1DCB9JPA>; Mon, 5 Feb 2001 15:53:33 -0500
Message-ID: <0F31E5C394DAD311B60C00E029101A07041014E4@corpmx9.isus.emc.com>
From: Black_David@emc.com
To: ips@ece.cmu.edu
Subject: Security Use Requirements
Date: Mon, 5 Feb 2001 15:53:30 -0500
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: text/plain
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
In Orlando, I picked up an action item to determine what
the requirements are for *use* of security features,
as opposed to requirements for *implementation*. I
believe the answer to be that it is acceptable to
specify security measures weaker than those one would
want to use in full generality on a public network,
where "weaker" includes no security.
There are two important caveats that apply:
- Security of the negotiation mechanism becomes
very important when this is done, as there's
an obvious man-in-the-middle attack on the
negotiation mechanism to get the endpoints
to negotiate weaker security than they intended.
- The weaker security mechanisms need to be documented
in terms of their security properties (and lack
thereof), as well as environments in which they
are appropriate. The "Security Considerations"
section of RFC 2338 (VRRP) has been recommended
as a good example of this.
--David
---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA 01748
+1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500
black_david@emc.com Mobile: +1 (978) 394-7754
---------------------------------------------------
From owner-ips@ece.cmu.edu Mon Feb 5 20:49:04 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA21912
for ; Mon, 5 Feb 2001 20:49:03 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f15NQu826276
for ips-outgoing; Mon, 5 Feb 2001 18:26:56 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gate.internaut.com ([64.38.134.108])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f15NQA626243
for ; Mon, 5 Feb 2001 18:26:11 -0500 (EST)
Received: from e1kj2 (e1kj2 [64.38.134.109])
by gate.internaut.com (8.10.2/8.10.2) with SMTP id f15NKJR12957;
Mon, 5 Feb 2001 15:20:23 -0800
From: "Bernard Aboba"
To: ,
Cc: "RJ Atkinson" ,
"Smb@Research. Att. Com"
Subject: RE: Security Use Requirements
Date: Mon, 5 Feb 2001 15:26:13 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
X-Mimeole: Produced By Microsoft MimeOLE V5.50.4133.2400
Importance: Normal
In-Reply-To: <0F31E5C394DAD311B60C00E029101A07041014E4@corpmx9.isus.emc.com>
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
It is hard for me to see how you could
get away with no security services at all
(e.g. no per-packet authentication and integrity
protection for iSCSI PDUs).
After all, we're talking about facilities
that are used by the world's major financial
institutions. If this data isn't worth protecting,
I don't know what is. Do you really want
attackers to be able to manipulate the contents
of bank accounts at will over the Internet?
Furthermore, there really isn't a sound technical
argument for dispensing with security. There are
chipsets available that can provide IPSEC
integrity and authentication services at
speeds of 1 Gbps or higher.
-----Original Message-----
From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
Black_David@emc.com
Sent: Monday, February 05, 2001 12:54 PM
To: ips@ece.cmu.edu
Subject: Security Use Requirements
In Orlando, I picked up an action item to determine what
the requirements are for *use* of security features,
as opposed to requirements for *implementation*. I
believe the answer to be that it is acceptable to
specify security measures weaker than those one would
want to use in full generality on a public network,
where "weaker" includes no security.
There are two important caveats that apply:
- Security of the negotiation mechanism becomes
very important when this is done, as there's
an obvious man-in-the-middle attack on the
negotiation mechanism to get the endpoints
to negotiate weaker security than they intended.
- The weaker security mechanisms need to be documented
in terms of their security properties (and lack
thereof), as well as environments in which they
are appropriate. The "Security Considerations"
section of RFC 2338 (VRRP) has been recommended
as a good example of this.
--David
---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA 01748
+1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500
black_david@emc.com Mobile: +1 (978) 394-7754
---------------------------------------------------
From owner-ips@ece.cmu.edu Mon Feb 5 21:46:25 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA23468
for ; Mon, 5 Feb 2001 21:46:24 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f160jqe29073
for ips-outgoing; Mon, 5 Feb 2001 19:45:52 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f160jB629026
for ; Mon, 5 Feb 2001 19:45:12 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f161t2073475;
Mon, 5 Feb 2001 17:55:02 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: "Y P Cheng" ,
"David Robinson" ,
Subject: RE: iSCSI Data Integrity - Digests
Date: Mon, 5 Feb 2001 16:43:31 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
In-Reply-To: <001701c08fba$b6cc3860$90c809c0@yp_portable.advansys.com>
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Y.P.
Forcing frame alignment does little to assist iSCSI if mid-tier equipment
removes such framing despite option negotiation. The urgent pointer has a
defined behavior within RFC documentation such that to define it as a record
boundary is not in keeping with standards. To implement and use an urgent
pointer as a record boundary pointer clearly modifies the TCP standard.
Requiring modification of all compliant TCP implementations is modifying the
standard. There are already several TCP negotiated options such as SACK and
no one suggested disallowing use of existing standard options. The urgent
pointer as a record boundary marker, as several people have indicated, is
not part of the standard, can not be done reliably with existing
implementations, and clearly places this concept into the non-standard
category.
PDU discovery can be done in a number of ways. If the SCTP structures were
adopted, then a pseudo-frame could be declared to start at fixed intervals
within the TCP byte stream. This could be done without modifying any TCP
implementations and as every device could support this mechanism, it could
be done in a uniform way. The SCTP structures allow encapsulation of larger
structures and could comply with this interval and define handshakes and
retries independent of TCP. This becomes required with added digest errors
that would need handling above TCP and hopefully below SCSI. I understand
the desire to use reserved bits and tweak behavior but with such a sizeable
install base, such desires only appear deceivingly simple to realize.
The NET-SCSI standard should be independent of the underlying transport.
There should be a transport layer added to TCP to provide all needed
features demanded by SAN. SCTP does a good job in defining these modern
transport requirements and an additional layer could be added to TCP to
provide a transport compliant to both SCTP using TCP. In doing so, there
would be virtually no difference between the two transports and there would
be little doubt about eventual migration.
Doug
> Doug,
>
> May be I am mistaken, but I don't think what I am suggesting is doing
> non-standard TCP. What is a non-standard TCP if I follow all the RFCs
> approved by IETF? What I think you are referring to the non-standard
> implementation, i.e., not working working with existing TCP stacks.
>
> The urgent pointer, 32-bit TCP window, one segment per PDU, even with the
> addition of SACK are all part of "standard" TCP, just not every TCP
> implementations are supporting them. This is why option negotiation comes
> in. I hope I am not mistaken.
>
> Y.P.
>
>
From owner-ips@ece.cmu.edu Mon Feb 5 21:46:26 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA23469
for ; Mon, 5 Feb 2001 21:46:25 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f161Iqt00379
for ips-outgoing; Mon, 5 Feb 2001 20:18:52 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from main.connectcom.net (ns1.advansys.com [204.247.22.3] (may be forged))
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f161IP600362
for ; Mon, 5 Feb 2001 20:18:25 -0500 (EST)
Received: from yp_portable (anubis.advansys.com [204.247.22.2]) by main.connectcom.net with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id 11CPQF7Y; Mon, 5 Feb 2001 17:18:23 -0800
From: "Y P Cheng"
To: "Douglas Otis" ,
"David Robinson" ,
Subject: RE: iSCSI Data Integrity - Digests
Date: Mon, 5 Feb 2001 17:16:04 -0800
Message-ID: <003d01c08fda$5ff1d2a0$90c809c0@yp_portable.advansys.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
X-MIMEOLE: Produced By Microsoft MimeOLE V4.72.3110.3
Importance: Normal
In-Reply-To:
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> Forcing frame alignment does little to assist iSCSI if mid-tier equipment
> removes such framing despite option negotiation.
I am not aware of any mid-tier box being TCP-aware and repackaging TCP
segments. If so, it must also involve in different TCP options including
SACK and 32-bit window, not to mention changing the TCP header and
recalculating checksum.
> The urgent pointer has a defined behavior within RFC documentation
> such that to define it as a record boundary is not in keeping
> with standards. To implement and use an urgent pointer as a record
> boundary pointer clearly modifies the TCP standard.
Very interesting argument. Urgent pointer is meant to be a inband signal
that can be used for any particular purpose a user has in mind. Saying
urgent pointer as a marker for a record or a message being a modification of
TCP standard, IMHO, is definitely a stretch. However, I yield the debate to
the TCP gurus.
From owner-ips@ece.cmu.edu Mon Feb 5 21:47:35 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA23489
for ; Mon, 5 Feb 2001 21:47:34 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f161Hra00350
for ips-outgoing; Mon, 5 Feb 2001 20:17:53 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from hotmail.com (f111.law4.hotmail.com [216.33.149.111])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f161Hh600338
for ; Mon, 5 Feb 2001 20:17:43 -0500 (EST)
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
Mon, 5 Feb 2001 17:17:37 -0800
Received: from 203.200.0.87 by lw4fd.law4.hotmail.msn.com with HTTP; Tue, 06 Feb 2001 01:17:37 GMT
X-Originating-IP: [203.200.0.87]
From: "Santosh Rao"
To: ips@ece.cmu.edu
Subject: RE: iSCSI : Abort Task "connection allegiance"
Date: Tue, 06 Feb 2001 01:17:37 -0000
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID:
X-OriginalArrivalTime: 06 Feb 2001 01:17:37.0343 (UTC) FILETIME=[977AE4F0:01C08FDA]
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
>There are two reasons for issuing abort: 1) application software exits and
>OS needs cleaning up, and 2) a request times out and it needs clean
>restart.
>In either cases, we don't want the abort arrives there before the
>one to be aborted. Hence, using zero CmdSN is unwise. If the abort gets
>there first,the target will complete the abort quickly and later executes
>the one to be aborted anyway.
YP,
"connection allegiance" implies the Abort Task is sent on the same
TCP connection as the command. By sending the Abort Task on the
same TCP connection, [and since it is sent after the command],
the ordering is preserved. The abort will not reach before the
command.
The only case where the abort would reach before the command
would be if the iSCSI layer at the target had discarded the
command PDU [due to an iSCSI layer error]. In such a case, since
the command will never be active at the target, the target
would respond to the Abort Task indicating "No Task Found"
or "Function Complete" which should be ok.
>
> > In the interests of keeping I/O timeout recovery quick, the
> > "connection allegiance" policy, currently applicable to all phases > of
>a command, should be extended to the Abort Task for that > command as well
>and the Abort Task be sent with a CmdSN of 0.
>
>I don't follow this logic at all. Are you trying to say that if the target
>gets an abort, it should enter connection allegiance until the one to be
>aborted arrives? This would real make a mess on the target microcode. How
>do we even know the aborted request would arrive? IMHO, abort should never
>have a zero CmdSN.
"Connection Allegiance" is enforced by an initiator [by sending the
command and the Abort Task on the same TCP connection.]. (This
terminology is curently used in the draft).
Connection Allegiance for a target impies all the PDUs for a command,
sent from a target to an initiator, should be sent on the same TCP
connection on which the command was received. (i.e. READ Data PDUs,
R2T, SCSI Response PDU, ..).
Regards,
Santosh
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com
From owner-ips@ece.cmu.edu Mon Feb 5 21:48:30 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id VAA23500
for ; Mon, 5 Feb 2001 21:48:30 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f161MC200562
for ips-outgoing; Mon, 5 Feb 2001 20:22:12 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from hotmail.com (f24.law4.hotmail.com [216.33.149.24])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f161LI600523
for ; Mon, 5 Feb 2001 20:21:18 -0500 (EST)
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
Mon, 5 Feb 2001 17:21:08 -0800
Received: from 203.200.0.87 by lw4fd.law4.hotmail.msn.com with HTTP; Tue, 06 Feb 2001 01:21:08 GMT
X-Originating-IP: [203.200.0.87]
From: "Santosh Rao"
To: ips@ece.cmu.edu
Subject: Re: Section 4.1 (Login Phase)
Date: Tue, 06 Feb 2001 01:21:08 -0000
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID:
X-OriginalArrivalTime: 06 Feb 2001 01:21:08.0878 (UTC) FILETIME=[159092E0:01C08FDB]
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Re-sending...
Julian,
Are you saying the Reject is NOT meant to be used for all format
errors (?)
How then is the following description of Reject PDU (Section 2.19) to
be interpreted :
"It may happen that a target receives a message with a format error
(inconsistent fields, reserved fields not 0, inexistent LUN etc.) or
a digest error (invalid payload or header). The target returns the
header of the message in error as the data of the response.
2.20 Reason
The reject Reason is coded as follows:
1 - Format Error
2 - Header Digest Error
3 - Payload Digest Error "
Why are 2 mechanisms required to indicate format errors ? The one in Login
response (reject with F=0) seems redundant.
Regards,
Santosh
_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com
From owner-ips@ece.cmu.edu Mon Feb 5 23:55:52 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA25647
for ; Mon, 5 Feb 2001 23:55:51 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f162bsM02990
for ips-outgoing; Mon, 5 Feb 2001 21:37:54 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f162b0602959
for ; Mon, 5 Feb 2001 21:37:01 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f163ko073556;
Mon, 5 Feb 2001 19:46:50 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: "Y P Cheng" ,
"David Robinson" ,
Subject: RE: iSCSI Data Integrity - Digests
Date: Mon, 5 Feb 2001 18:35:19 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
In-Reply-To: <003d01c08fda$5ff1d2a0$90c809c0@yp_portable.advansys.com>
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Y.P.
> > The urgent pointer has a defined behavior within RFC documentation
> > such that to define it as a record boundary is not in keeping
> > with standards. To implement and use an urgent pointer as a record
> > boundary pointer clearly modifies the TCP standard.
>
> Very interesting argument. Urgent pointer is meant to be a inband signal
> that can be used for any particular purpose a user has in mind. Saying
> urgent pointer as a marker for a record or a message being a
> modification of
> TCP standard, IMHO, is definitely a stretch. However, I yield
> the debate to
> the TCP gurus.
The urgent pointer marks urgent data and must coalesce to include additional
urgent data. To use the urgent pointer to mark boundaries in close
proximity, as close as the next segment as you described, then you will be
required to modify TCP to not coalesce. Starving send will not be conducive
to accelerated use and not coalescing the pointer is a violation of the TCP
standard. Both TCP send and receive coalesce. Yes, the urgent pointer is
an inband signal but with these clearly defined limitations. These
limitations make it impractical for use as a record marker and likely to
thwart TCP Offload Engines as it also requires unique data handling. The
urgent pointer is not a general purpose marker. Its devised use is to
expedite and only requires a single variable. You are attempting to expand
use of the urgent pointer to encompass a greatly revised definition
requiring an attribute reserved for every message unit up to one per
segment. It is definitely a stretch to consider this not a modification of
the TCP standard, IMO.
Doug
From owner-ips@ece.cmu.edu Mon Feb 5 23:56:16 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id XAA25658
for ; Mon, 5 Feb 2001 23:56:11 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f162qs003489
for ips-outgoing; Mon, 5 Feb 2001 21:52:54 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f162q0603455
for ; Mon, 5 Feb 2001 21:52:00 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f1642I073572;
Mon, 5 Feb 2001 20:02:19 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: "Santosh Rao" ,
Subject: RE: iSCSI : Abort Task "connection allegiance"
Date: Mon, 5 Feb 2001 18:50:47 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
In-Reply-To:
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Santosh,
> YP,
>
> "connection allegiance" implies the Abort Task is sent on the same
> TCP connection as the command. By sending the Abort Task on the
> same TCP connection, [and since it is sent after the command],
> the ordering is preserved. The abort will not reach before the
> command.
>
> The only case where the abort would reach before the command
> would be if the iSCSI layer at the target had discarded the
> command PDU [due to an iSCSI layer error]. In such a case, since
> the command will never be active at the target, the target
> would respond to the Abort Task indicating "No Task Found"
> or "Function Complete" which should be ok.
The advantage of multiple connections or multiple homing is removed if
connection allegiance is mandated. The ability to recover from a seemingly
failed connection may include a request to restart using a different
connection. Such request will likely be sent over a different connection if
there is a desire to migrate. Connection allegiance was to allow
distributed state information to remain isolated. This isolation is
problematic and not mandating allegiance ensures there will always be a
means to communicate intermediate states between connections. Connection
allegiance should only be preferred.
Doug
From owner-ips@ece.cmu.edu Tue Feb 6 04:11:39 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id EAA10308
for ; Tue, 6 Feb 2001 04:11:38 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f166xuE13313
for ips-outgoing; Tue, 6 Feb 2001 01:59:56 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f166xH613287
for ; Tue, 6 Feb 2001 01:59:17 -0500 (EST)
Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23])
by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id HAA10168;
Tue, 6 Feb 2001 07:58:54 +0100
From: julian_satran@il.ibm.com
Received: from d12mta05.de.ibm.com (d12mta06_cs0 [9.165.222.255])
by d12relay02.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id HAA46874;
Tue, 6 Feb 2001 07:58:35 +0100
Received: by d12mta05.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569EB.002652D6 ; Tue, 6 Feb 2001 07:58:35 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: "Bernard Aboba"
cc: Black_David@emc.com, ips@ece.cmu.edu, "RJ Atkinson" ,
"Smb@Research. Att. Com" , biran@il.ibm.com
Message-ID:
Date: Tue, 6 Feb 2001 08:54:03 +0200
Subject: RE: Security Use Requirements
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Bernard,
The enquiry David took upon himself to make is that if we have a strong
requirement to have digests that include authentication and integrity as a
minimal requirement or if we could work with a sequence (listed here in
order "increasing" integrity/security) that looks like:
1-none
2-data integrity(CRC) and/or authentication of the parties at session start
3-full security through TLS and transport-IPSec
The answer to this seems to be (as we expected) that as long as the
negotiation is done in a proper way eliminating the possibility of getting
drawn down to a weak security scheme when at least one of the parties wants
a higher level we can go with the outlined sequence of schemes.
The additional point we will have to ponder is where we want to draw the
line for a "minimal compliant" iSCSI. The current (true for June 2000)
consensus between the authors was "implementation - somewhere within 2 and
deployment at 1" - with CRCs mandatory to implement (optional to use) and
all the rest is optional to use and implement.
Considering the spectrum of devices and applications for iSCSI I think that
this is a reasonable choice.
Regards,
Julo
"Bernard Aboba" on 06/02/2001 01:26:13
Please respond to "Bernard Aboba"
To: Black_David@emc.com, ips@ece.cmu.edu
cc: "RJ Atkinson" , "Smb@Research. Att. Com"
Subject: RE: Security Use Requirements
It is hard for me to see how you could
get away with no security services at all
(e.g. no per-packet authentication and integrity
protection for iSCSI PDUs).
After all, we're talking about facilities
that are used by the world's major financial
institutions. If this data isn't worth protecting,
I don't know what is. Do you really want
attackers to be able to manipulate the contents
of bank accounts at will over the Internet?
Furthermore, there really isn't a sound technical
argument for dispensing with security. There are
chipsets available that can provide IPSEC
integrity and authentication services at
speeds of 1 Gbps or higher.
-----Original Message-----
From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
Black_David@emc.com
Sent: Monday, February 05, 2001 12:54 PM
To: ips@ece.cmu.edu
Subject: Security Use Requirements
In Orlando, I picked up an action item to determine what
the requirements are for *use* of security features,
as opposed to requirements for *implementation*. I
believe the answer to be that it is acceptable to
specify security measures weaker than those one would
want to use in full generality on a public network,
where "weaker" includes no security.
There are two important caveats that apply:
- Security of the negotiation mechanism becomes
very important when this is done, as there's
an obvious man-in-the-middle attack on the
negotiation mechanism to get the endpoints
to negotiate weaker security than they intended.
- The weaker security mechanisms need to be documented
in terms of their security properties (and lack
thereof), as well as environments in which they
are appropriate. The "Security Considerations"
section of RFC 2338 (VRRP) has been recommended
as a good example of this.
--David
---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA 01748
+1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500
black_david@emc.com Mobile: +1 (978) 394-7754
---------------------------------------------------
From owner-ips@ece.cmu.edu Tue Feb 6 10:48:29 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id KAA18921
for ; Tue, 6 Feb 2001 10:48:29 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f16DGsU06673
for ips-outgoing; Tue, 6 Feb 2001 08:16:54 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from uucp1.nwnexus.com (uucp1.nwnexus.com [206.63.63.110])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f16DGZH06656
for ; Tue, 6 Feb 2001 08:16:35 -0500 (EST)
Received: from internaut.com (uucp@localhost)
by uucp1.nwnexus.com (8.8.8/8.8.8) with UUCP id FAA00945;
Tue, 6 Feb 2001 05:16:26 -0800 (PST)
Received: by internaut.com (NX5.67e/NeXT-3.0)
id AA00350; Tue, 6 Feb 01 05:52:53 -0800
Date: Tue, 6 Feb 2001 05:52:52 -0800 (GMT-0800)
From: "Bernard D. Aboba"
To: julian_satran@il.ibm.com
Cc: Black_David@emc.com, ips@ece.cmu.edu, RJ Atkinson ,
"Smb@Research. Att. Com" , biran@il.ibm.com
Subject: RE: Security Use Requirements
In-Reply-To:
Message-Id:
Mime-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
> deployment at 1" - with CRCs mandatory to implement (optional to use) and
> all the rest is optional to use and implement.
CRCs only provide integrity protection, but not authentication since they
are not keyed. Thus, it provides no protection against spoofing
attacks. Even if the CRC is non-linear, it is not hard to build
a device that will change packets on the fly without fear of detection. The
TCP checksum is non-linear but it can be guessed right about half the
time.
An example of the kinds of attacks that are possible is found at:
http://www.isaac.cs.berkeley.edu/isaac/wep-faq.html. I'm sure the
folks at Berkeley will be happy to provide an equivalent analysis for
iSCSI.
Do you really want to enable attackers to insert or change data destined
a SAN disk at will? Even if the iSCSI SAN is using linklocal addressing,
and therefore is not accessible from the Internet, there is still risk from
internal attack.
A more reasonable approach would be to require at least authentication
and integrity protection (e.g. IPSEC AH or ESP null).
From owner-ips@ece.cmu.edu Tue Feb 6 14:11:01 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA28284
for ; Tue, 6 Feb 2001 14:11:01 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f16GjrW17862
for ips-outgoing; Tue, 6 Feb 2001 11:45:53 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f16GjWH17837
for ; Tue, 6 Feb 2001 11:45:32 -0500 (EST)
Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22])
by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id RAA109412;
Tue, 6 Feb 2001 17:45:14 +0100
From: julian_satran@il.ibm.com
Received: from d12mta05.de.ibm.com (d12mta06_cs0 [9.165.222.255])
by d12relay01.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id RAA93446;
Tue, 6 Feb 2001 17:45:02 +0100
Received: by d12mta05.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569EB.005C02D6 ; Tue, 6 Feb 2001 17:45:00 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: "Bernard D. Aboba"
cc: Black_David@emc.com, ips@ece.cmu.edu, RJ Atkinson ,
"Smb@Research. Att. Com" , biran@il.ibm.com
Message-ID:
Date: Tue, 6 Feb 2001 18:32:20 +0200
Subject: RE: Security Use Requirements
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Bernard,
Both IPSec and TLS will be in the standard. As we are talking about
speeds that will be in excess of 1GBs even on modest disk controllers we
where all hesitant if to make anything in this category mandatory to
implement today.
We assume that all those who require security beyond CRC and session
authentication will pay for and get it. However those that build a Storage
Area Newtork within a small enterprise completely isolated from the
internet will not have to pay for what they do not need.
Regards,
Julo
"Bernard D. Aboba" on 06/02/2001 15:52:52
Please respond to "Bernard D. Aboba"
To: Julian Satran/Haifa/IBM@IBMIL
cc: Black_David@emc.com, ips@ece.cmu.edu, RJ Atkinson ,
"Smb@Research. Att. Com" , Ofer
Biran/Haifa/IBM@IBMIL
Subject: RE: Security Use Requirements
> deployment at 1" - with CRCs mandatory to implement (optional to use) and
> all the rest is optional to use and implement.
CRCs only provide integrity protection, but not authentication since they
are not keyed. Thus, it provides no protection against spoofing
attacks. Even if the CRC is non-linear, it is not hard to build
a device that will change packets on the fly without fear of detection. The
TCP checksum is non-linear but it can be guessed right about half the
time.
An example of the kinds of attacks that are possible is found at:
http://www.isaac.cs.berkeley.edu/isaac/wep-faq.html. I'm sure the
folks at Berkeley will be happy to provide an equivalent analysis for
iSCSI.
Do you really want to enable attackers to insert or change data destined
a SAN disk at will? Even if the iSCSI SAN is using linklocal addressing,
and therefore is not accessible from the Internet, there is still risk from
internal attack.
A more reasonable approach would be to require at least authentication
and integrity protection (e.g. IPSEC AH or ESP null).
From owner-ips@ece.cmu.edu Tue Feb 6 14:12:53 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id OAA28331
for ; Tue, 6 Feb 2001 14:12:52 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f16GHpx16277
for ips-outgoing; Tue, 6 Feb 2001 11:17:51 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from hotmail.com (f32.law4.hotmail.com [216.33.149.32])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f16GGxH16238
for ; Tue, 6 Feb 2001 11:16:59 -0500 (EST)
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
Tue, 6 Feb 2001 08:16:53 -0800
Received: from 203.200.0.72 by lw4fd.law4.hotmail.msn.com with HTTP; Tue, 06 Feb 2001 16:16:53 GMT
X-Originating-IP: [203.200.0.72]
From: "Santosh Rao"
To: ips@ece.cmu.edu
Subject: RE: iSCSI : Abort Task "connection allegiance"
Date: Tue, 06 Feb 2001 21:46:53 +0530
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID:
X-OriginalArrivalTime: 06 Feb 2001 16:16:53.0499 (UTC) FILETIME=[37DAECB0:01C09058]
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
>Santosh,
>The advantage of multiple connections or multiple homing is removed if
>connection allegiance is mandated.
Doug,
Section 1.2.5 already mandates connection allegiance for all PDUs of a
command.
("For SCSI commands that require data and/or parameter
transfer, the (optional) data and the status for a command must be
sent over the same TCP connection that was used to deliver the SCSI
command (we call this "connection allegiance").")
I believe the dis-advantages of sending different PDUs of a command over
different connections has already been discussed and discarded due to the
various state sharing issues as well as out-of-order issues it causes.
So, is the concern expressed about the existing MUST in the above statement
in the draft ? Or are the comments directed towards the proposal to extend
connection allegiance to the Abort Task as well ?
>The ability to recover from a seemingly failed connection may include a
>request to restart using a different connection. Such request will likely
>be sent over a different connection if there is a desire to migrate.
>Connection allegiance was to allow distributed state information to remain
>isolated. This isolation is problematic and not mandating allegiance
>ensures there will always be a means to communicate intermediate states
>between connections.
While the "retry" command processing at the target may involve co-operation
across multiple NIC instances to fetch the data/status, this should be ok
since it is the exception path. We should not attempt to optimize this path
and affect the mainline paths. The mainline I/O paths require connection
allegiance for a number of reasons that have been previously discussed on
the reflector.
>Connection allegiance should only be preferred.
The lack of connection allegiance for all PDus of a command can cause
out-of-order problems, with status PDUs arriving at the initiator ahead of
the data PDUs.It also requires sharing I/O state information across NICs for
the mainline paths, as compared to the current model where this is only
required in exception paths.
Regards,
Santosh
_________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.
From owner-ips@ece.cmu.edu Tue Feb 6 17:06:11 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA03359
for ; Tue, 6 Feb 2001 17:06:11 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f16J1xC25502
for ips-outgoing; Tue, 6 Feb 2001 14:01:59 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gnat.inet.org (209-9-249-62.sdsl.cais.net [209.9.249.62])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f16GvhH18508
for ; Tue, 6 Feb 2001 11:57:43 -0500 (EST)
Received: from mosquito.inet.org (mosquito.inet.org [10.30.20.240])
by gnat.inet.org (Postfix) with ESMTP
id 1B1B18264F; Tue, 6 Feb 2001 11:56:12 -0500 (EST)
Message-Id: <5.0.0.25.2.20010206114429.009ff9d0@10.30.15.2>
X-Sender: rja@10.30.15.2
X-Mailer: QUALCOMM Windows Eudora Version 5.0
Date: Tue, 06 Feb 2001 11:47:49 -0500
To: julian_satran@il.ibm.com
From: RJ Atkinson
Subject: RE: Security Use Requirements
Cc: "Bernard D. Aboba" , Black_David@emc.com,
ips@ece.cmu.edu, RJ Atkinson , ,
biran@il.ibm.com
In-Reply-To:
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
At 11:32 06/02/01, julian_satran@il.ibm.com wrote:
>Bernard,
>
>Both IPSec and TLS will be in the standard. As we are talking about
>speeds that will be in excess of 1GBs even on modest disk controllers we were all hesitant if to make anything in this category mandatory to
>implement today.
I suspect that failing to make security mandatory-to-
implement will tend to prevent the specification(s) from
ever becoming IETF standards-track RFC(s). However, the WG
can certainly *try* to slide by.
As a potential user, I would prefer that security be
made mandatory-to-implement. Inevitably stuff intended for
private networks ends up on the global Internet by accident
or out of expediency. Further, even on private networks
there are significant insider security risks.
Ran
rja@inet.org
From owner-ips@ece.cmu.edu Tue Feb 6 17:10:20 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id RAA03498
for ; Tue, 6 Feb 2001 17:10:15 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f16J6tx25823
for ips-outgoing; Tue, 6 Feb 2001 14:06:55 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gateway.sanlight.org (adsl-63-202-160-80.dsl.snfc21.pacbell.net [63.202.160.80])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f16J5qH25768
for ; Tue, 6 Feb 2001 14:05:52 -0500 (EST)
Received: from ljoy ([10.0.0.18])
by gateway.sanlight.org (8.11.0/8.11.0) with SMTP id f16KGD074692;
Tue, 6 Feb 2001 12:16:13 -0800 (PST)
(envelope-from dotis@sanlight.net)
From: "Douglas Otis"
To: "Santosh Rao" ,
Subject: RE: iSCSI : Abort Task "connection allegiance"
Date: Tue, 6 Feb 2001 11:04:43 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0)
Importance: Normal
In-Reply-To:
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
Santosh,
It is common to have multiple routers providing access over different IPs to
the same machine. At an indication of trouble, an alternative path would be
employed to prevent any significant service disruption with a path switch
the extent of effort. The present scheme only considers multiple
connections as a means to modify TCP congestion algorithms in obtaining
larger shares of the network bandwidth and ignores issues of reliability and
disruption avoidance. To require ULP (higher level protocol) exchange as
part of making the path switch on existing connections together with an ULP
restart of service is highly disruptive and indicates an unreliable delivery
scheme.
Already all PDUs are marked with a serial number across all connections.
The problems of out-of-order issues occur due to a lack of integrity of this
serialization. When this serialization fails, it is hoped TCP provides
necessary sequential assurance and is thus imposing a connection restriction
as this would then introduce a different underlying TCP sequence. The
present handshake already mandates some state sharing across all connections
with Exp* and Max*. Allowing fail-over at the transport level is the only
means that would have a broader application and would not impose
restrictions on the ULP such as SCSI. Such an approach is a means of
isolating various layers involved and a means of ensuring there is not a
required cooperation of the ULP.
Although you express concern about main-line execution of SCSI where to
reduce the amount of state information shared between connections,
connection allegiance is employed; the means of allowing a transition to a
different connection without assistance of SCSI should be allowed for a good
design. Any required state information should be transferable
automatically. The alternative is highly disruptive. As a performance
optimization, indicate preference for allegiance but do not mandate it and
repair serialization integrity of the transport to remain viable across
multiple connections. To do less will reduce the integrity of the system
and impose highly complex scenarios for recovery.
Doug
> >Santosh,
> >The advantage of multiple connections or multiple homing is removed if
> >connection allegiance is mandated.
>
> Doug,
>
> Section 1.2.5 already mandates connection allegiance for all PDUs of a
> command.
> ("For SCSI commands that require data and/or parameter
> transfer, the (optional) data and the status for a command must be
> sent over the same TCP connection that was used to deliver the SCSI
> command (we call this "connection allegiance").")
>
> I believe the dis-advantages of sending different PDUs of a command over
> different connections has already been discussed and discarded due to the
> various state sharing issues as well as out-of-order issues it causes.
>
> So, is the concern expressed about the existing MUST in the above
> statement
> in the draft ? Or are the comments directed towards the proposal
> to extend
> connection allegiance to the Abort Task as well ?
>
> >The ability to recover from a seemingly failed connection may include a
> >request to restart using a different connection. Such request
> will likely
> >be sent over a different connection if there is a desire to migrate.
> >Connection allegiance was to allow distributed state information
> to remain
> >isolated. This isolation is problematic and not mandating allegiance
> >ensures there will always be a means to communicate intermediate states
> >between connections.
>
> While the "retry" command processing at the target may involve
> co-operation
> across multiple NIC instances to fetch the data/status, this should be ok
> since it is the exception path. We should not attempt to optimize
> this path
> and affect the mainline paths. The mainline I/O paths require connection
> allegiance for a number of reasons that have been previously discussed on
> the reflector.
>
> >Connection allegiance should only be preferred.
>
> The lack of connection allegiance for all PDus of a command can cause
> out-of-order problems, with status PDUs arriving at the initiator
> ahead of
> the data PDUs.It also requires sharing I/O state information
> across NICs for
> the mainline paths, as compared to the current model where this is only
> required in exception paths.
>
> Regards,
> Santosh
>
>
> _________________________________________________________________________
> Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com.
>
>
From owner-ips@ece.cmu.edu Tue Feb 6 18:40:54 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA04635
for ; Tue, 6 Feb 2001 18:40:53 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f16LN3j03816
for ips-outgoing; Tue, 6 Feb 2001 16:23:03 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from e34.bld.us.ibm.com (e34.co.us.ibm.com [32.97.110.132])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f16LMjH03801
for ; Tue, 6 Feb 2001 16:22:46 -0500 (EST)
Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.132.205])
by e34.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id QAA91072;
Tue, 6 Feb 2001 16:06:37 -0500
Received: from f5n70e (d03nm094h.boulder.ibm.com [9.99.140.70])
by westrelay02.boulder.ibm.com (8.8.8m3/NCO v4.95) with ESMTP id OAA153896;
Tue, 6 Feb 2001 14:22:08 -0700
X-Priority: 1 (High)
Importance: Normal
Subject: RE: Security Use Requirements
To: "Bernard Aboba"
Cc: , , "RJ Atkinson" ,
"Smb@Research. Att. Com"
X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000
Message-ID:
From: "John Hufferd"
Date: Tue, 6 Feb 2001 13:21:56 -0800
X-MIMETrack: Serialize by Router on D03NM094/03/M/IBM(Release 5.0.6 |December 14, 2000) at
02/06/2001 02:22:08 PM
MIME-Version: 1.0
Content-type: text/plain; charset=us-ascii
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Bernard,
I think that Julian addressed this, but, an installation might want only
the connection to the local environment, and if so administratively tell
the iSCSI ends to not do the encryption etc. Especially if some of the
ends are Laptops and Desktops. But all side must implement the features.
By the way you might have slightly overstated the IPSec chips going at full
gig speed, when you talk about triple Des. And if there are some they are
not within the normal costs one would expect for a iSCSI NIC HBA.
.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
(408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Internet address: hufferd@us.ibm.com
"Bernard Aboba" @ece.cmu.edu on 02/05/2001 03:26:13 PM
Sent by: owner-ips@ece.cmu.edu
To: ,
cc: "RJ Atkinson" , "Smb@Research. Att. Com"
Subject: RE: Security Use Requirements
It is hard for me to see how you could
get away with no security services at all
(e.g. no per-packet authentication and integrity
protection for iSCSI PDUs).
After all, we're talking about facilities
that are used by the world's major financial
institutions. If this data isn't worth protecting,
I don't know what is. Do you really want
attackers to be able to manipulate the contents
of bank accounts at will over the Internet?
Furthermore, there really isn't a sound technical
argument for dispensing with security. There are
chipsets available that can provide IPSEC
integrity and authentication services at
speeds of 1 Gbps or higher.
-----Original Message-----
From: owner-ips@ece.cmu.edu [mailto:owner-ips@ece.cmu.edu]On Behalf Of
Black_David@emc.com
Sent: Monday, February 05, 2001 12:54 PM
To: ips@ece.cmu.edu
Subject: Security Use Requirements
In Orlando, I picked up an action item to determine what
the requirements are for *use* of security features,
as opposed to requirements for *implementation*. I
believe the answer to be that it is acceptable to
specify security measures weaker than those one would
want to use in full generality on a public network,
where "weaker" includes no security.
There are two important caveats that apply:
- Security of the negotiation mechanism becomes
very important when this is done, as there's
an obvious man-in-the-middle attack on the
negotiation mechanism to get the endpoints
to negotiate weaker security than they intended.
- The weaker security mechanisms need to be documented
in terms of their security properties (and lack
thereof), as well as environments in which they
are appropriate. The "Security Considerations"
section of RFC 2338 (VRRP) has been recommended
as a good example of this.
--David
---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA 01748
+1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500
black_david@emc.com Mobile: +1 (978) 394-7754
---------------------------------------------------
From owner-ips@ece.cmu.edu Tue Feb 6 18:44:00 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA04673
for ; Tue, 6 Feb 2001 18:43:55 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f16LOwB03928
for ips-outgoing; Tue, 6 Feb 2001 16:24:58 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from main.connectcom.net (ns1.advansys.com [204.247.22.3] (may be forged))
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f16LOAH03879
for ; Tue, 6 Feb 2001 16:24:10 -0500 (EST)
Received: from yp_portable (anubis.advansys.com [204.247.22.2]) by main.connectcom.net with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21)
id 11CPQG83; Tue, 6 Feb 2001 13:07:20 -0800
From: "Y P Cheng"
To: "John Hufferd"
Cc: "'Ips@Ece. Cmu. Edu'"
Subject: RE: iSCSI Data Integrity - Digests
Date: Tue, 6 Feb 2001 13:05:07 -0800
Message-ID: <000901c09080$7bc07d00$90c809c0@yp_portable.advansys.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook 8.5, Build 4.71.2173.0
In-Reply-To:
Importance: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V4.72.3110.3
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
> With my TC Hat on:
> YP and others,
> It does not matter if you are technically right. We have been over and
> over this. Please let it lay. If you are sending something that is
> technically TCP/IP but has some extra (legal) trick. That is of course
> fine, if you are talking to your own adapter. We here in the IETF would
> not like to talk about your tricks to your adapter. We need to have
> knowledge that works with other vendors HW and SW Targets. That means we
> have to define how to use the tricks, and exploit them. As you know we
> have been told Over and Over that the IETF as a whole will NOT accept this
> kind of thing in our draft standard, so we should not waste our time and
> reflector bandwidth talking about it.
>
> If you folks just have to talk about it, please take it off the reflector.
I apologize for giving you the impression of wanting to talk about something
not appropriate on this reflector. As you know, I wrote in responding to
comments on non-Standard TCP implementation. The real question is that each
time a new TCP option is added, does the old TCP remains as Standard or the
new TCP becomes a new Standard. I stated that this WG was prohibited from
discussing this topic. But, many people in this WG does not seem to realize
that iSCSI chips will have new TCP features and options whether we like it
or not. While this reflector doesn't wish to talk about TCP, it certainly
could not demand that all new TCP implementations for iSCSI are prohibited.
Peace!
From owner-ips@ece.cmu.edu Tue Feb 6 18:45:35 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA04700
for ; Tue, 6 Feb 2001 18:45:30 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f16LKwX03705
for ips-outgoing; Tue, 6 Feb 2001 16:20:58 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gate.internaut.com ([64.38.134.108])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f16LKlH03690
for ; Tue, 6 Feb 2001 16:20:47 -0500 (EST)
Received: from e1kj2 (e1kj2 [64.38.134.109])
by gate.internaut.com (8.10.2/8.10.2) with SMTP id f16LFER20188;
Tue, 6 Feb 2001 13:15:15 -0800
From: "Bernard Aboba"
To:
Cc: , , "RJ Atkinson" ,
"Smb@Research. Att. Com" ,
Subject: RE: Security Use Requirements
Date: Tue, 6 Feb 2001 13:21:14 -0800
Message-ID:
MIME-Version: 1.0
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
In-Reply-To:
X-Mimeole: Produced By Microsoft MimeOLE V5.50.4133.2400
Importance: Normal
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Content-Transfer-Encoding: 7bit
>Both IPSec and TLS will be in the standard.
Including *both* IPSEC and TLS as options will probably result
in both much higher cost to customers as well as fewer vendors
implementing either. Choosing one or the other (IPSEC is my
preference) would be a better choice.
In general, it is a bad idea to have too many security options in
a standard. Options are bad because you have to implement them
to be able to interoperate with those people who do, yet you cannot
guarantee that the work will be used. In particular, including both
IPSEC and TLS would cause vendors to try to implement both on
the same NIC, since they could never be sure which security scheme
would be used on a given iSCSI target. This would drive up the cost
enormously since the acceleration architectures are very different
between the protocols.
>As we are talking about speeds that will be in excess of 1GBs
>even on modest disk controllers we were all hesitant if to make
>anything in this category mandatory to implement today.
As Ran noted, making implementation mandatory doesn't mean that
users have to turn it on. But not making it mandatory will probably
ensure that security will not be available in most cases. Given
the modest cost of adding IPSEC acceleration hardware (today
IPSEC accelerated NICs are available for around $100) it is hard
to see how this would be a worthwhile tradeoff.
Right now, SSL/TLS acceleration hardware is a lot more expensive than
IPSEC accelerators (at least $2000), so I could understand if SSL/TLS
were left out of the standard.
>We assume that all those who require security beyond CRC
CRC provides only integrity protection, but since it is not keyed,
there is no per-packet authentication or protection against spoofing
or modification. CRC therefore has little or no security value.
> session authentication
All session authentication does is guarantee the user identity
at the beginning of the session. Unless you have per-packet
authentication and integrity protection, all bets are off after
that. In particular, session authentication provides no protection
against TCP session hijacking.
>However those that build a Storage Area Newtork within a
>small enterprise completely isolated from the
>internet will not have to pay for what they do not need.
Actually, the decision made above will guarantee that all users
will need to pay a huge incremental cost for security, because
of the over abundance of security options. In reality, given the
projected cost of 1+ Gbps iSCSI accelerated adapters, the incremetal
cost of implementing IPSEC security would be minimal.
From owner-ips@ece.cmu.edu Tue Feb 6 18:46:01 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id SAA04712
for ; Tue, 6 Feb 2001 18:45:59 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f16MG2E06790
for ips-outgoing; Tue, 6 Feb 2001 17:16:02 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from gnat.inet.org (209-9-249-62.sdsl.cais.net [209.9.249.62])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f16LeOH04789
for ; Tue, 6 Feb 2001 16:40:24 -0500 (EST)
Received: from mosquito.inet.org (mosquito.inet.org [10.30.20.240])
by gnat.inet.org (Postfix) with ESMTP
id 817ED8264F; Tue, 6 Feb 2001 16:38:00 -0500 (EST)
Message-Id: <5.0.0.25.2.20010206162519.009ea9c0@10.30.15.2>
X-Sender: rja@10.30.15.2
X-Mailer: QUALCOMM Windows Eudora Version 5.0
X-Priority: 1 (Highest)
Date: Tue, 06 Feb 2001 16:29:38 -0500
To: "John Hufferd"
From: RJ Atkinson
Subject: RE: Security Use Requirements
Cc: "Bernard Aboba" , ,
, "RJ Atkinson" ,
"Smb@Research. Att. Com"
In-Reply-To:
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
At 16:21 06/02/01, John Hufferd wrote:
>I think that Julian addressed this, but, an installation might want only
>the connection to the local environment, and if so administratively tell
>the iSCSI ends to not do the encryption etc. Especially if some of the
>ends are Laptops and Desktops. But all side must implement the features.
Implement != turn on operationally. The above explains
why clever vendors might have a configuration knob to turn off
security. The above does NOT make any kind of case for not
always *implementing* security.
>By the way you might have slightly overstated the IPSec chips going at full
>gig speed, when you talk about triple Des. And if there are some they are
>not within the normal costs one would expect for a iSCSI NIC HBA.
So if you believe the costs are so high, implement single DES.
For a lot of threat environments DES-CBC is sufficient and it surely beats the hell out of nothing. By the way, the crypto parts vendors
that I'm talking with must be giving me better prices than you,
which I find surprising, since by the parts quotes I'm seeing
Bernard's math works just fine.
Nothing anyone has said here has given any kind of reasonable
excuse to not make implementing security mandatory. There has been
lots of rationale for making it optional for the user to turn on
for a given box, but nothing for making it optional to implement.
(Implement in the box != deploy operationally).
Ran
rja@inet.org
From owner-ips@ece.cmu.edu Tue Feb 6 20:35:23 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA05795
for ; Tue, 6 Feb 2001 20:35:22 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f16NY0x10900
for ips-outgoing; Tue, 6 Feb 2001 18:34:00 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from mxbh4.isus.emc.com (mxbh4.isus.emc.com [168.159.208.52])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f16NWxH10833
for ; Tue, 6 Feb 2001 18:32:59 -0500 (EST)
Received: by mxbh4.isus.emc.com with Internet Mail Service (5.5.2650.21)
id <1DCB0HJ0>; Tue, 6 Feb 2001 18:32:51 -0500
Message-ID: <0F31E5C394DAD311B60C00E029101A07041014F1@corpmx9.isus.emc.com>
From: Black_David@emc.com
To: rja@inet.org, julian_satran@il.ibm.com
Cc: aboba@internaut.com, ips@ece.cmu.edu, biran@il.ibm.com
Subject: RE: Security Use Requirements
Date: Tue, 6 Feb 2001 18:32:10 -0500
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2650.21)
Content-Type: text/plain
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Some related clarifications:
> I suspect that failing to make security mandatory-to-
> implement will tend to prevent the specification(s) from
> ever becoming IETF standards-track RFC(s). However, the WG
> can certainly *try* to slide by.
My original post was about mandatory-to-use, NOT
mandatory-to-implement. As for making security not
mandatory-to-implement ... NOT on my watch ... and
remember that Steve Bellovin is one of the ips WG
co-chairs, making me a "good cop" by comparison on
this sort of issue ;-).
> Another customary IETF position is to work on the basis that
> all IETF protocols are likely (in practice) to be deployed over
> The Global Internet. So the threat model discussions normally
> MUST include those kind of threats and can't be limited to threats
> on a private intranet.
That is also correct ... and I say that with my WG co-chair
hat firmly in place. There is also language to this
effect in the ips WG charter.
It's unfortunate that Julian focussed only on integrity
in his response. Going back to the Pittsburgh (summer
2000) minutes, secure authentication will be mandatory to
implement ... so sayeth the AD. I also believe that some
form of cryptographically authenticated integrity will
likewise be mandatory to implement (otherwise it's too
easy to hijack an authenticated connection). This means
signed cryptographic checksums, not just CRCs (as Bernard
explained). OTOH, it's not clear that confidentiality
will be mandatory to implement.
The whole point on mandatory-to-implement vs. mandatory-to-use
is to adapt available security technologies to the circumstances.
One simple example is that confidentiality is NOT mandatory-to-use
in IPsec, otherwise AH wouldn't exist ... and 5 demerits to anyone
who uses that comment as a jumping off point to open the debate
about whether AH should exist at all on this list.
I'm not sure that the statement that both IPsec and TLS
will be in the spec is the rough consensus of the WG at
this point. I also want to caution folks that simply
requiring these technologies to be implemented is not
sufficient to address all the security issues - one example
for iSCSI is that something will need to be said about
the relationship of target names to the identities used
for IPsec and/or TLS authentication. Let me point everyone
to Jeff Schiller's security tutorial from the San Diego
IETF meeting at: http://jis.mit.edu/sectutorial/ which
covers this and a bunch of related material on security
at a level that should be accessible to almost everyone.
Thanks,
--David
---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA 01748
+1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500
black_david@emc.com Mobile: +1 (978) 394-7754
---------------------------------------------------
From owner-ips@ece.cmu.edu Tue Feb 6 20:35:37 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id UAA05806
for ; Tue, 6 Feb 2001 20:35:36 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f16NZ1t10942
for ips-outgoing; Tue, 6 Feb 2001 18:35:01 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f16NXwH10892
for ; Tue, 6 Feb 2001 18:33:58 -0500 (EST)
Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2448.0)
id ; Tue, 6 Feb 2001 15:33:48 -0800
Message-ID:
From: Joshua Tseng
To: Bernard Aboba
Cc: ips@ece.cmu.edu
Subject: RE: Security Use Requirements
Date: Tue, 6 Feb 2001 15:33:40 -0800
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="windows-1252"
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Bernard,
> Including *both* IPSEC and TLS as options will probably result
> in both much higher cost to customers as well as fewer vendors
> implementing either. Choosing one or the other (IPSEC is my
> preference) would be a better choice.
IPSec and TLS provide different capabilities. While IPSec is
easier to implement and hence, as you point out, much cheaper,
TLS affords the capability of securely communicating through
proxys. On the other hand, IPSec is a real pain to get through
firewalls. If we decide that iSCSI doesn't need to go through
firewalls, then we could let go of TLS. But the last I remember
from reading the IPS charter, it is still a requirement.
>
> In general, it is a bad idea to have too many security options in
> a standard. Options are bad because you have to implement them
> to be able to interoperate with those people who do, yet you cannot
> guarantee that the work will be used. In particular, including both
> IPSEC and TLS would cause vendors to try to implement both on
> the same NIC, since they could never be sure which security scheme
> would be used on a given iSCSI target. This would drive up the cost
> enormously since the acceleration architectures are very different
> between the protocols.
Why should IPSEC or TLS be mandatory for iSCSI when it isn't for http,
dns, and other network protocols? I'm aware of nothing inherent in
iSCSI that makes it need special security measures any more than those
other protocols. If the security folks intended for all protocols to
be secure, wouldn't it be more efficient for the security WG to say it
just once, and mandate it for all IP protocols?
Josh
>
> >As we are talking about speeds that will be in excess of 1GBs
> >even on modest disk controllers we were all hesitant if to make
> >anything in this category mandatory to implement today.
>
> As Ran noted, making implementation mandatory doesn't mean that
> users have to turn it on. But not making it mandatory will probably
> ensure that security will not be available in most cases. Given
> the modest cost of adding IPSEC acceleration hardware (today
> IPSEC accelerated NICs are available for around $100) it is hard
> to see how this would be a worthwhile tradeoff.
>
> Right now, SSL/TLS acceleration hardware is a lot more expensive than
> IPSEC accelerators (at least $2000), so I could understand if SSL/TLS
> were left out of the standard.
>
> >We assume that all those who require security beyond CRC
>
> CRC provides only integrity protection, but since it is not keyed,
> there is no per-packet authentication or protection against spoofing
> or modification. CRC therefore has little or no security value.
>
> > session authentication
>
> All session authentication does is guarantee the user identity
> at the beginning of the session. Unless you have per-packet
> authentication and integrity protection, all bets are off after
> that. In particular, session authentication provides no protection
> against TCP session hijacking.
>
> >However those that build a Storage Area Newtork within a
> >small enterprise completely isolated from the
> >internet will not have to pay for what they do not need.
>
> Actually, the decision made above will guarantee that all users
> will need to pay a huge incremental cost for security, because
> of the over abundance of security options. In reality, given the
> projected cost of 1+ Gbps iSCSI accelerated adapters, the incremetal
> cost of implementing IPSEC security would be minimal.
>
From owner-ips@ece.cmu.edu Tue Feb 6 22:25:41 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id WAA08966
for ; Tue, 6 Feb 2001 22:25:40 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f171N2M15994
for ips-outgoing; Tue, 6 Feb 2001 20:23:02 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from server1.NishanSystems.COM (smtp.nishansystems.com [216.217.36.162])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f171MwH15988
for ; Tue, 6 Feb 2001 20:22:58 -0500 (EST)
Received: by smtp.nishansystems.com with Internet Mail Service (5.5.2448.0)
id ; Tue, 6 Feb 2001 17:22:48 -0800
Message-ID:
From: Joshua Tseng
To: Black_David@emc.com
Cc: ips@ece.cmu.edu
Subject: RE: Security Use Requirements
Date: Tue, 6 Feb 2001 17:22:46 -0800
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2448.0)
Content-Type: text/plain;
charset="windows-1252"
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
David,
Question: Does your "mandatory-to-implement" mean
"mandatory-to-implement-on-the-same-box", or
"mandatory-to-implement-on-the-same-or-different-box"?
:-)
IPSec security gateways are widely available now, from
many different vendors. Are you ruling out their use
to fulfill the security requirement?
Josh
> -----Original Message-----
> From: Black_David@emc.com [mailto:Black_David@emc.com]
> Sent: Tuesday, February 06, 2001 3:32 PM
> To: rja@inet.org; julian_satran@il.ibm.com
> Cc: aboba@internaut.com; ips@ece.cmu.edu; biran@il.ibm.com
> Subject: RE: Security Use Requirements
>
>
> Some related clarifications:
>
> > I suspect that failing to make security mandatory-to-
> > implement will tend to prevent the specification(s) from
> > ever becoming IETF standards-track RFC(s). However, the WG
> > can certainly *try* to slide by.
>
> My original post was about mandatory-to-use, NOT
> mandatory-to-implement. As for making security not
> mandatory-to-implement ... NOT on my watch ... and
> remember that Steve Bellovin is one of the ips WG
> co-chairs, making me a "good cop" by comparison on
> this sort of issue ;-).
>
> > Another customary IETF position is to work on the basis that
> > all IETF protocols are likely (in practice) to be deployed over
> > The Global Internet. So the threat model discussions normally
> > MUST include those kind of threats and can't be limited to threats
> > on a private intranet.
>
> That is also correct ... and I say that with my WG co-chair
> hat firmly in place. There is also language to this
> effect in the ips WG charter.
>
> It's unfortunate that Julian focussed only on integrity
> in his response. Going back to the Pittsburgh (summer
> 2000) minutes, secure authentication will be mandatory to
> implement ... so sayeth the AD. I also believe that some
> form of cryptographically authenticated integrity will
> likewise be mandatory to implement (otherwise it's too
> easy to hijack an authenticated connection). This means
> signed cryptographic checksums, not just CRCs (as Bernard
> explained). OTOH, it's not clear that confidentiality
> will be mandatory to implement.
>
> The whole point on mandatory-to-implement vs. mandatory-to-use
> is to adapt available security technologies to the circumstances.
> One simple example is that confidentiality is NOT mandatory-to-use
> in IPsec, otherwise AH wouldn't exist ... and 5 demerits to anyone
> who uses that comment as a jumping off point to open the debate
> about whether AH should exist at all on this list.
>
> I'm not sure that the statement that both IPsec and TLS
> will be in the spec is the rough consensus of the WG at
> this point. I also want to caution folks that simply
> requiring these technologies to be implemented is not
> sufficient to address all the security issues - one example
> for iSCSI is that something will need to be said about
> the relationship of target names to the identities used
> for IPsec and/or TLS authentication. Let me point everyone
> to Jeff Schiller's security tutorial from the San Diego
> IETF meeting at: http://jis.mit.edu/sectutorial/ which
> covers this and a bunch of related material on security
> at a level that should be accessible to almost everyone.
>
> Thanks,
> --David
> ---------------------------------------------------
> David L. Black, Senior Technologist
> EMC Corporation, 42 South St., Hopkinton, MA 01748
> +1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500
> black_david@emc.com Mobile: +1 (978) 394-7754
> ---------------------------------------------------
>
From owner-ips@ece.cmu.edu Wed Feb 7 03:39:45 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA26724
for ; Wed, 7 Feb 2001 03:39:44 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f177Y7K01712
for ips-outgoing; Wed, 7 Feb 2001 02:34:07 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d06lmsgate-3.uk.ibm.com (d06lmsgate-3.uk.ibm.com [195.212.29.3])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f177WvH01673
for ; Wed, 7 Feb 2001 02:32:58 -0500 (EST)
Received: from d06relay01.portsmouth.uk.ibm.com (d06relay01.portsmouth.uk.ibm.com [9.166.84.147])
by d06lmsgate-3.uk.ibm.com (1.0.0) with ESMTP id HAA193328;
Wed, 7 Feb 2001 07:24:13 GMT
From: julian_satran@il.ibm.com
Received: from d06mta03.portsmouth.uk.ibm.com (d06mta03_cs0 [9.180.35.1])
by d06relay01.portsmouth.uk.ibm.com (8.8.8m3/NCO v4.95) with SMTP id HAA115674;
Wed, 7 Feb 2001 07:32:44 GMT
Received: by d06mta03.portsmouth.uk.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id 802569EC.00297113 ; Wed, 7 Feb 2001 07:32:39 +0000
X-Lotus-FromDomain: IBMIL@IBMDE@IBMGB
To: Black_David@emc.com
cc: rja@inet.org, aboba@internaut.com, ips@ece.cmu.edu, biran@il.ibm.com
Message-ID: <802569EC.00296FDB.00@d06mta03.portsmouth.uk.ibm.com>
Date: Wed, 7 Feb 2001 09:28:04 +0200
Subject: RE: Security Use Requirements
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
David,
In Orlando the agreement was that authentication digests can be left to
specialized protocols (IPsec and TLS) and iSCSI
is not mandated to have them specified outside such a protocol. That was
what I asked for and there was agreement on that
waiting for your input from the security area.
That is why I focused on the basic thing we have to specify (rather than
refer to) - basic integrity and session authentication.
As for mandatory to implement or not the question Josh raised is very much
on all our minds - if it is external to our environment (like IPsec) what
should we make mandatory to implement. IMHO we can make mandatory to
implement the kickoff of a security engine (as we have in the current
draft). BTW this is also the way security is implemented in all the
widespread protocols today - HTTP has a TLS upgrade command, Telnet and FTP
have security kickoffs etc.. NFS has nothing more either. None of them
specifies anything beyond that as the security component selected is the
one that is dictating what is the minimum and what the module is offering.
We choose to include both TLS and IPsec as they have different capabilities
(as Josh has stated already).
All the above are architectural considerations and I hope we did not make
any major blunder there.
The issue you raised - can now be translated should we make IPsec or TLS
mandatory to implement?
I think we should not.
Regards,
Julo
Black_David@emc.com on 07/02/2001 01:32:10
Please respond to Black_David@emc.com
To: rja@inet.org, Julian Satran/Haifa/IBM@IBMIL
cc: aboba@internaut.com, ips@ece.cmu.edu, Ofer Biran/Haifa/IBM@IBMIL
Subject: RE: Security Use Requirements
Some related clarifications:
> I suspect that failing to make security mandatory-to-
> implement will tend to prevent the specification(s) from
> ever becoming IETF standards-track RFC(s). However, the WG
> can certainly *try* to slide by.
My original post was about mandatory-to-use, NOT
mandatory-to-implement. As for making security not
mandatory-to-implement ... NOT on my watch ... and
remember that Steve Bellovin is one of the ips WG
co-chairs, making me a "good cop" by comparison on
this sort of issue ;-).
> Another customary IETF position is to work on the basis that
> all IETF protocols are likely (in practice) to be deployed over
> The Global Internet. So the threat model discussions normally
> MUST include those kind of threats and can't be limited to threats
> on a private intranet.
That is also correct ... and I say that with my WG co-chair
hat firmly in place. There is also language to this
effect in the ips WG charter.
It's unfortunate that Julian focussed only on integrity
in his response. Going back to the Pittsburgh (summer
2000) minutes, secure authentication will be mandatory to
implement ... so sayeth the AD. I also believe that some
form of cryptographically authenticated integrity will
likewise be mandatory to implement (otherwise it's too
easy to hijack an authenticated connection). This means
signed cryptographic checksums, not just CRCs (as Bernard
explained). OTOH, it's not clear that confidentiality
will be mandatory to implement.
The whole point on mandatory-to-implement vs. mandatory-to-use
is to adapt available security technologies to the circumstances.
One simple example is that confidentiality is NOT mandatory-to-use
in IPsec, otherwise AH wouldn't exist ... and 5 demerits to anyone
who uses that comment as a jumping off point to open the debate
about whether AH should exist at all on this list.
I'm not sure that the statement that both IPsec and TLS
will be in the spec is the rough consensus of the WG at
this point. I also want to caution folks that simply
requiring these technologies to be implemented is not
sufficient to address all the security issues - one example
for iSCSI is that something will need to be said about
the relationship of target names to the identities used
for IPsec and/or TLS authentication. Let me point everyone
to Jeff Schiller's security tutorial from the San Diego
IETF meeting at: http://jis.mit.edu/sectutorial/ which
covers this and a bunch of related material on security
at a level that should be accessible to almost everyone.
Thanks,
--David
---------------------------------------------------
David L. Black, Senior Technologist
EMC Corporation, 42 South St., Hopkinton, MA 01748
+1 (508) 435-1000 x75140 FAX: +1 (508) 497-8500
black_david@emc.com Mobile: +1 (978) 394-7754
---------------------------------------------------
From owner-ips@ece.cmu.edu Wed Feb 7 03:39:56 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id DAA26735
for ; Wed, 7 Feb 2001 03:39:55 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f176k7K29929
for ips-outgoing; Wed, 7 Feb 2001 01:46:07 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate.de.ibm.com (d12lmsgate.de.ibm.com [195.212.91.199])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f176jpH29915
for ; Wed, 7 Feb 2001 01:45:51 -0500 (EST)
Received: from d12relay02.de.ibm.com (d12relay02.de.ibm.com [9.165.215.23])
by d12lmsgate.de.ibm.com (1.0.0) with ESMTP id HAA68394;
Wed, 7 Feb 2001 07:45:41 +0100
From: julian_satran@il.ibm.com
Received: from d12mta05.de.ibm.com (d12mta06_cs0 [9.165.222.255])
by d12relay02.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id HAA89120;
Wed, 7 Feb 2001 07:45:23 +0100
Received: by d12mta05.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569EC.00251D92 ; Wed, 7 Feb 2001 07:45:24 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: "Bernard Aboba"
cc: Black_David@emc.com, ips@ece.cmu.edu, "RJ Atkinson" ,
"Smb@Research. Att. Com" , biran@il.ibm.com
Message-ID:
Date: Wed, 7 Feb 2001 08:08:23 +0200
Subject: RE: Security Use Requirements
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Bernard,
This is becoming an interesting thread.
The reason we went after TLS is that it can be used for session
authentication with stronger schemes than
and it is very popular for software implementations.
As for the cost of the hardware - the figures you quote are for 100Mbs (and
even there the NIC numbers are higher).
The low-end iSCSI adapters will cost well under $100 (at 1GBbs).
I don't envision all the options becoming necessary for hardware
implementations. The pieces we wanted from TLS can be implemented in
software.
If we where forced to select one I would too go for IPsec (and that is what
we have in the current draft) but then we have
to specify session authentication on our own and keep updating it as new
schemes enter the world. It seems far better
to rely on a specialized standard and TLS fits the bill.
Regards,
Julo
"Bernard Aboba" on 06/02/2001 23:21:14
Please respond to "Bernard Aboba"
To: Julian Satran/Haifa/IBM@IBMIL
cc: Black_David@emc.com, ips@ece.cmu.edu, "RJ Atkinson" ,
"Smb@Research. Att. Com" , Ofer
Biran/Haifa/IBM@IBMIL
Subject: RE: Security Use Requirements
>Both IPSec and TLS will be in the standard.
Including *both* IPSEC and TLS as options will probably result
in both much higher cost to customers as well as fewer vendors
implementing either. Choosing one or the other (IPSEC is my
preference) would be a better choice.
In general, it is a bad idea to have too many security options in
a standard. Options are bad because you have to implement them
to be able to interoperate with those people who do, yet you cannot
guarantee that the work will be used. In particular, including both
IPSEC and TLS would cause vendors to try to implement both on
the same NIC, since they could never be sure which security scheme
would be used on a given iSCSI target. This would drive up the cost
enormously since the acceleration architectures are very different
between the protocols.
>As we are talking about speeds that will be in excess of 1GBs
>even on modest disk controllers we were all hesitant if to make
>anything in this category mandatory to implement today.
As Ran noted, making implementation mandatory doesn't mean that
users have to turn it on. But not making it mandatory will probably
ensure that security will not be available in most cases. Given
the modest cost of adding IPSEC acceleration hardware (today
IPSEC accelerated NICs are available for around $100) it is hard
to see how this would be a worthwhile tradeoff.
Right now, SSL/TLS acceleration hardware is a lot more expensive than
IPSEC accelerators (at least $2000), so I could understand if SSL/TLS
were left out of the standard.
>We assume that all those who require security beyond CRC
CRC provides only integrity protection, but since it is not keyed,
there is no per-packet authentication or protection against spoofing
or modification. CRC therefore has little or no security value.
> session authentication
All session authentication does is guarantee the user identity
at the beginning of the session. Unless you have per-packet
authentication and integrity protection, all bets are off after
that. In particular, session authentication provides no protection
against TCP session hijacking.
>However those that build a Storage Area Newtork within a
>small enterprise completely isolated from the
>internet will not have to pay for what they do not need.
Actually, the decision made above will guarantee that all users
will need to pay a huge incremental cost for security, because
of the over abundance of security options. In reality, given the
projected cost of 1+ Gbps iSCSI accelerated adapters, the incremetal
cost of implementing IPSEC security would be minimal.
From owner-ips@ece.cmu.edu Wed Feb 7 05:47:34 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id FAA27521
for ; Wed, 7 Feb 2001 05:47:33 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f178tDd04600
for ips-outgoing; Wed, 7 Feb 2001 03:55:13 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate-2.de.ibm.com (d12lmsgate-2.de.ibm.com [195.212.91.200])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f178sMH04570
for ; Wed, 7 Feb 2001 03:54:22 -0500 (EST)
Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22])
by d12lmsgate-2.de.ibm.com (1.0.0) with ESMTP id JAA90492;
Wed, 7 Feb 2001 09:54:15 +0100
From: julian_satran@il.ibm.com
Received: from d12mta05.de.ibm.com (d12mta06_cs0 [9.165.222.255])
by d12relay01.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id JAA70096;
Wed, 7 Feb 2001 09:54:11 +0100
Received: by d12mta05.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569EC.0030E748 ; Wed, 7 Feb 2001 09:54:09 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: Glen Turner
cc: Black_David@emc.com, rja@inet.org, aboba@internaut.com, ips@ece.cmu.edu,
biran@il.ibm.com
Message-ID:
Date: Wed, 7 Feb 2001 10:49:27 +0200
Subject: Re: Security Use Requirements (and security issue with iSCSI boot
deployment)
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
Glen,
You are right and we are specifically considering doing something special
(and mandatory to implement -:)) for boot.
Regards,
Julo
Glen Turner on 07/02/2001 10:10:48
Please respond to Glen Turner
To: Black_David@emc.com
cc: rja@inet.org, Julian Satran/Haifa/IBM@IBMIL, aboba@internaut.com,
ips@ece.cmu.edu, Ofer Biran/Haifa/IBM@IBMIL
Subject: Re: Security Use Requirements (and security issue with iSCSI boot
deployment)
Black_David@emc.com wrote:
>
> The whole point on mandatory-to-implement vs. mandatory-to-use
> is to adapt available security technologies to the circumstances.
For example, some of us run huge public anonymous FTP/HTTP
servers that mirror software. They spend a lot of their time
serving Linux CD images. There is a strong performance and
usability case for doing this serving using iSCSI.
The images are typically GPG-signed by their originators,
which addresses the core security issue for the users of the
CD image: did someone hack the server and alter the image?
The assurance that can be provided iSCSI's security mechanisms
simply can't address this question, and thus we may as well
take any performance advantage from disabling them and offer
the server's users an increased maximum user count.
---------------------------------
Note that iSCSI-boot doesn't yet provide a mechanism for end-to-end
checking that a disk image is unaltered and thus is particularly prone
to serving a hacked boot image when the server has been compromised.
At the least it should be noted in Security Considerations that vendors
should consider providing a mechanism for vendor-to-booter verification
of a boot image.
It would be really nice if iSCSI-boot suggested a mechanism, so that
it could be built into ROMs by manufacturers that are implementing
iSCSI-boot and so that the hardware manufacturer could not use the
mechanism to lock out alternative operating systems.
Given the random-access nature of iSCSI this could be tricky to
implement;
GPG-signing each disk block might not be the best for performance :-)
Nevertheless, some mechanism would be useful, otherwise iSCSI-boot
becomes
a neat mechanism with which to implement an enterprise-wide compromise.
Glen.
From owner-ips@ece.cmu.edu Wed Feb 7 06:02:01 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id GAA27686
for ; Wed, 7 Feb 2001 06:02:01 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f179eAP06223
for ips-outgoing; Wed, 7 Feb 2001 04:40:10 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from e34.bld.us.ibm.com (e34.co.us.ibm.com [32.97.110.132])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f179dnH06208
for ; Wed, 7 Feb 2001 04:39:49 -0500 (EST)
Received: from westrelay02.boulder.ibm.com (westrelay02.boulder.ibm.com [9.99.132.205])
by e34.bld.us.ibm.com (8.9.3/8.9.3) with ESMTP id EAA127624;
Wed, 7 Feb 2001 04:24:10 -0500
Received: from f5n70e (d03nm094h.boulder.ibm.com [9.99.140.70])
by westrelay02.boulder.ibm.com (8.8.8m3/NCO v4.95) with ESMTP id CAA57204;
Wed, 7 Feb 2001 02:39:47 -0700
X-Priority: 1 (High)
Importance: Normal
Subject: RE: Security Use Requirements
To: RJ Atkinson
Cc:
X-Mailer: Lotus Notes Release 5.0.3 (Intl) 21 March 2000
Message-ID:
From: "John Hufferd"
Date: Wed, 7 Feb 2001 01:37:05 -0800
X-MIMETrack: Serialize by Router on D03NM094/03/M/IBM(Release 5.0.6 |December 14, 2000) at
02/07/2001 02:39:47 AM
MIME-Version: 1.0
Content-type: text/plain; charset=us-ascii
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
RJ,
I can not tell, from the obvious intensity of your note, if you agree with
me are not. As Julian pointed out the prices were probably for the 100
Mb/s links. In any event. the need is for security is at least 3DES.
Also the cost of a Gigabit chip for 3DES, I just found out, is $300 for
Samples. If you go from Cost to street price, you will probably have a 3x
difference. Now, that is a bit much for only one of the key componets on
an iSCSI/TOE HBA that is suppose to be competitive to Fibre Channels.
Therefore, I am reconsidering my statements, that we Must Implement. Your
opinion might be useful here. I had thought that in the real world,
separate box IPSec/ TLS, at the edge of the secure network might be all
that is required.
Since we must also consider the SW implementations on Desktops and Laptops
perhaps IPSec is OK there, since the volume of data will be relatively
slow.
Now, I am beginning to think that it is reasonable for one of the following
approaches to be OK. That is, one of those approaches should meet the
requirement for "Must Implement".
1. Only implementing an interface to the external IPSec/TLS box
2, SW implementation of IPSec/TLS
3. HW IPSec/TLS
In this case, if the customer wants protection, they either pay for the
separate IPSec/TLS box (can be shared by many connections), or the HW
IPSec/TLS or accept the overhead and lack of speed in SW implementations.
OK folks, lets here all your opinions on this.
.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
(408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Internet address: hufferd@us.ibm.com
RJ Atkinson @ece.cmu.edu on 02/06/2001 01:29:38 PM
Sent by: owner-ips@ece.cmu.edu
To: John Hufferd/San Jose/IBM@IBMUS
cc: "Bernard Aboba" , ,
, "RJ Atkinson" , "Smb@Research.
Att. Com"
Subject: RE: Security Use Requirements
At 16:21 06/02/01, John Hufferd wrote:
>I think that Julian addressed this, but, an installation might want only
>the connection to the local environment, and if so administratively tell
>the iSCSI ends to not do the encryption etc. Especially if some of the
>ends are Laptops and Desktops. But all side must implement the features.
Implement != turn on operationally. The above explains
why clever vendors might have a configuration knob to turn off
security. The above does NOT make any kind of case for not
always *implementing* security.
>By the way you might have slightly overstated the IPSec chips going at
full
>gig speed, when you talk about triple Des. And if there are some they are
>not within the normal costs one would expect for a iSCSI NIC HBA.
So if you believe the costs are so high, implement single DES.
For a lot of threat environments DES-CBC is sufficient and it surely beats
the hell out of nothing. By the way, the crypto parts vendors
that I'm talking with must be giving me better prices than you,
which I find surprising, since by the parts quotes I'm seeing
Bernard's math works just fine.
Nothing anyone has said here has given any kind of reasonable
excuse to not make implementing security mandatory. There has been
lots of rationale for making it optional for the user to turn on
for a given box, but nothing for making it optional to implement.
(Implement in the box != deploy operationally).
Ran
rja@inet.org
From owner-ips@ece.cmu.edu Wed Feb 7 08:30:40 2001
Received: from ece.cmu.edu (ECE.CMU.EDU [128.2.236.200])
by ietf.org (8.9.1a/8.9.1a) with SMTP id IAA29018
for ; Wed, 7 Feb 2001 08:30:39 -0500 (EST)
Received: (from majordom@localhost)
by ece.cmu.edu (8.11.0/8.10.2) id f17BOAE22246
for ips-outgoing; Wed, 7 Feb 2001 06:24:10 -0500 (EST)
X-Authentication-Warning: ece.cmu.edu: majordom set sender to owner-ips@ece.cmu.edu using -f
Received: from d12lmsgate-3.de.ibm.com (d12lmsgate-3.de.ibm.com [195.212.91.201])
by ece.cmu.edu (8.11.0/8.10.2) with ESMTP id f17BO2H22237
for ; Wed, 7 Feb 2001 06:24:02 -0500 (EST)
Received: from d12relay01.de.ibm.com (d12relay01.de.ibm.com [9.165.215.22])
by d12lmsgate-3.de.ibm.com (1.0.0) with ESMTP id MAA35922;
Wed, 7 Feb 2001 12:23:53 +0100
From: julian_satran@il.ibm.com
Received: from d12mta05.de.ibm.com (d12mta06_cs0 [9.165.222.255])
by d12relay01.de.ibm.com (8.8.8m3/NCO v4.95) with SMTP id MAA108428;
Wed, 7 Feb 2001 12:23:49 +0100
Received: by d12mta05.de.ibm.com(Lotus SMTP MTA v4.6.5 (863.2 5-20-1999)) id C12569EC.003E9BB6 ; Wed, 7 Feb 2001 12:23:50 +0100
X-Lotus-FromDomain: IBMIL@IBMDE
To: "John Hufferd"
cc: RJ Atkinson , ips@ece.cmu.edu
Message-ID:
Date: Wed, 7 Feb 2001 13:19:16 +0200
Subject: RE: Security Use Requirements
Mime-Version: 1.0
Content-type: text/plain; charset=us-ascii
Content-Disposition: inline
Sender: owner-ips@ece.cmu.edu
Precedence: bulk
It would be interesting to findout if the chips support transport IPsec or
tunneling IPsec or both.
For tunneling IPsec you have to be a "believer" - i.e., you'll have to
believe that there is such a think on the wire
as iSCSI is oblivious to it. However if provided in an external box it
can provide excellent security.
The "required to implement" statement - if made by iSCSI - would have only
propaganda value
as it can't be assessed in a compliance testing suite.
Julo
"John Hufferd" on 07/02/2001 11:37:05
Please respond to "John Hufferd"
To: RJ Atkinson
cc: ips@ece.cmu.edu
Subject: RE: Security Use Requirements
RJ,
I can not tell, from the obvious intensity of your note, if you agree with
me are not. As Julian pointed out the prices were probably for the 100
Mb/s links. In any event. the need is for security is at least 3DES.
Also the cost of a Gigabit chip for 3DES, I just found out, is $300 for
Samples. If you go from Cost to street price, you will probably have a 3x
difference. Now, that is a bit much for only one of the key componets on
an iSCSI/TOE HBA that is suppose to be competitive to Fibre Channels.
Therefore, I am reconsidering my statements, that we Must Implement. Your
opinion might be useful here. I had thought that in the real world,
separate box IPSec/ TLS, at the edge of the secure network might be all
that is required.
Since we must also consider the SW implementations on Desktops and Laptops
perhaps IPSec is OK there, since the volume of data will be relatively
slow.
Now, I am beginning to think that it is reasonable for one of the following
approaches to be OK. That is, one of those approaches should meet the
requirement for "Must Implement".
1. Only implementing an interface to the external IPSec/TLS box
2, SW implementation of IPSec/TLS
3. HW IPSec/TLS
In this case, if the customer wants protection, they either pay for the
separate IPSec/TLS box (can be shared by many connections), or the HW
IPSec/TLS or accept the overhead and lack of speed in SW implementations.
OK folks, lets here all your opinions on this.
.
.
.
John L. Hufferd
Senior Technical Staff Member (STSM)
IBM/SSG San Jose Ca
(408) 256-0403, Tie: 276-0403, eFax: (408) 904-4688
Internet address: hufferd@us.ibm.com
RJ Atkinson @ece.cmu.edu on 02/06/2001 01:29:38 PM
Sent by: owner-ips@ece.cmu.edu
To: John Hufferd/San Jose/IBM@IBMUS
cc: "Bernard Aboba" , ,
, "RJ Atkinson" , "Smb@Research.
Att. Com"