Subject: [rbridge] Disable transit for a VLAN was (RE: Shared VLAN learning: What is it and why should wecare?)

From: sgai at nuovasystems.com (Silvano Gai)
Date: Sun, 1 Apr 2007 07:53:47 -0700

I changed the subject because I want us to clarify one of the
confusions:

"It is not possible to configure an RBridge to act as a transit node for
a particular VLAN".

In other words: "an RBridge that does not have any port/node in VLAN A
MUST act as a transit for VLAN A if other RBridges require so".

This is different from IEEE 802.1Q in which you can decide which switch
act as transit for which VLAN.

-- Silvano

> -----Original Message-----
> From: rbridge-bounces at postel.org [mailto:rbridge-bounces at postel.org]
On
> Behalf Of Radia Perlman
> Sent: Saturday, March 31, 2007 11:40 PM
> To: Russ White
> Cc: rbridge at postel.org
> Subject: Re: [rbridge] Shared VLAN learning: What is it and why should
> wecare?
> 
> I don't understand a lot of what's being said on this thread, and I
> suspect everyone is
> talking about subtly different solutions to a fairly esoteric set of
> somewhat related problems (sharing
> an IP router, sharing addresses in an IP subnet, NAT'ting VLAN IDs,
> ...). As Dale Carder mentioned,
> and gave us some pointers, there are several somewhat overlapping
> proposals floating around the industry.
> I believe there isn't a single cleanly stated problem, or a single
> cleanly statable solution.
> 
> So TRILL should do something. Whatever it is, I want to be able to
> understand what it is that RBridges should do.
> 
> But to answer Russ's questions (see at bottom of this note) -- the
> current RBridge protocol
> spec assumes that *all* RBridges filter based on VLAN tag.
> So an RBridge in the core, even if it doesn't attach to VLAN A, knows
> which other RBridges do. This
> allows efficient distribution of VLAN A traffic, for instance, the
IS-IS
> announcements of VLAN A endnode
> information.
> 
> The reason we need to let all the RBridges know what VLANs are in a
> group, say {Z, A, B, C, D},
> where Z is the primary, is so that an RBridge in
> the core knows that something tagged with VLAN Z needs to be
transmitted
> to all links in any of the
> 5 VLAN tags {Z, A, B, C, D} and something tagged with one of the
> secondaries, say A, needs to go to
> A and Z.
> 
> An example -- suppose there is an IP router R on VLAN Z (the primary).
> When R does an ARP for an
> endnode "d" in the IP subnet, R has no idea about VLANs, and the
RBridge
> RB1 that R attaches to, although
> RB1 might know that there's a grouping of VLANs, doesn't know which
VLAN
> to transmit the ARP to.
> Perhaps RB1 could duplicate broadcast packets tagged with Z, sending a
> copy to each of {Z, A, B, C, D}.
> 
> So RBridges in the core have to know to send the ARP along the
> distribution tree to all ports that are
> in any of the VLANs in the set.
> 
> In theory we could ignore VLAN tags, except at the final hop, and
then,
> as Russ said, it
> would be just an optimization for the RBridges in the core to know
about
> the groupings. But I still think things would be weird
> if the final RBridges had inconsistent configurations of the VLAN
> groupings, so I think it's a good idea to
> make sure all the RBridges agree on the groupings.
> 
> To summarize, it's good for them to agree because of three reasons:
> a) we can do optimal delivery of multicasts, filtering within the core
> rather than just at the final hop
> b) we detect misconfiguration
> c) we allow more central configuration (we don't need to configure all
> the RBridges, though that's a bit
> dangerous if all the RBridges that have been configured go down, since
> suddenly the groupings might disappear)
> 
> Radia
> 
> 
> 
> As for Russ's last questions:
> 
> >> 3. Does this have a larger impact on multicast, or smaller?
> 
> I don't understand the question, though given that he put a smiley
face
> in, perhaps
> it was a joke. I'm hoping that someone, tomorrow, will inform me that
all
> this
> SVL/FID stuff was a joke...
> 
> Radia
> 
> 
> 
> 
> Russ White wrote:
> 
> >-----BEGIN PGP SIGNED MESSAGE-----
> >Hash: SHA1
> >
> >
> >
> >
> >
> >>To avoid requiring configuration of all RBridges with these FIDs,
and
> >>still allow multicast filtering, I propose we have RBridges
advertise,
> >>in their IS-IS core instance, FIDs that they are configured with. So
at
> >>least one RBridge will say "Hey guys,
> >>I think VLANs Z, A, B, C, and D form a FID with primary=Z" and the
other
> >>RBridges will, I guess
> >>believe him.
> >>
> >>
> >
> >I'm trying to sort through this entire thing, and I wanted to ask:
> >
> >1. The reason we'd want to configure this on all rbridges would be to
> >optimize traffic flow?
> >
> >2. If 1 is true, and we didn't do the configuration on all rbridges,
how
> >much less efficient would the rbridge network be?
> >
> >3. Does this have a larger impact on multicast, or smaller?
> >
> >:-)
> >
> >Russ
> >
> >
> >-----BEGIN PGP SIGNATURE-----
> >Version: GnuPG v1.4.3 (Darwin)
> >Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
> >
> >iD8DBQFGDVS6ER27sUhU9OQRAnbCAJ99um0k9+At9V+z97fksfH4gUVoGACgw7Mg
> >DSuMdrc10XgRmx8w4PdN/f4=
> >=ItDw
> >-----END PGP SIGNATURE-----
> >
> >
> 
> _______________________________________________
> rbridge mailing list
> rbridge at postel.org
> http://mailman.postel.org/mailman/listinfo/rbridge

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Sun, 1 Apr 2007 08:07:37 -0700

The other reason for confusion is that SVL exists also without Private
VLANs.

IVL and SVL are internal switch operation mode. Se IEEE 802.1Q-2005
Section 7.4 and the begging of Appendix B. IMHO they don't pose any
particular requirement on the IP routers. Each VLAN maps to a router
sub-interface, that maps to an IP subnet.

SVL is used by Private VLANs, but there is much more to Private VLANs
that just SVL. IEEE 802.1Q has the similar concept of Asymmetric VLANs,
see EEE 802.1Q-2005 B.1.3 Asymmetric VLANs.

In future posting, please clarify if what you are stating applies in
general to SVL or it is specific of Private VLANs.

-- Silvano

On
IS-IS
transmitted
RBridge
VLAN
then,
about
face
all
and
advertise,
at
other
how

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Sun, 1 Apr 2007 08:52:37 -0700

I said it wrong, sorry, retrying:

"It is not possible to configure an RBridge to NOT act as a transit node
for a particular VLAN".

There is only one topology in RBridges, i.e. the core topology. In
Spanning Tree potentially there are many active topologies. In RBridge
the per VLAN instance only acts as "VLAN Pruning".

-- Silvano

On
> Behalf Of Silvano Gai
> Sent: Sunday, April 01, 2007 7:54 AM
> To: Radia Perlman; Russ White
> Subject: [rbridge] Disable transit for a VLAN was (RE: Shared VLAN
> learning:What is it and why should wecare?)
> 
>

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Mon, 2 Apr 2007 14:05:30 -0500

Radia,

	Let me try to re-state your explanation to see if I understand 
it...

	You believe that the problem that shared VLAN learning is used
to solve is conservation of IP addresses used in a single IP subnet, 
and that it is necessary to use shared VLAN learning to keep routers 
(in the subnet) from having to re-learn MAC+IP associations across a
set of VLANs in that same IP subnet, is that correct?

Thanks!

--
Eric Gray
Principal Engineer
Ericsson  

> From: rbridge-bounces at postel.org 
> [mailto:rbridge-bounces at postel.org] On Behalf Of Radia Perlman
> Sent: Thursday, March 29, 2007 2:56 AM
> To: rbridge at postel.org
> Subject: [rbridge] Shared VLAN learning: What is it and why 
> should we care?
> 
> Hopefully this explanation will be clear enough for people to 
> at least 
> figure out whether we
> are all talking about the same thing. Of course, feel free to 
> correct me 
> if my understanding is wrong.
> 
> First we need to understand what problem it is trying to 
> solve. This is 
> my understanding of that:
> 
> THE PROBLEM
> ------------------
> 
> Someone has a block of IP addresses to divvy up among
> a set of different organizations. Let's say the address block has
> 100 addresses, and there are 4 organizations. One strategy might be
> to give each organization 1/4 of the IP addresses each, and then 
> possibly even
> wind up having separate IP subnets (if the addresses can be assigned
> to be in different prefixes).
> 
> The problem, though, is that you don't know in advance how 
> many addresses
> each organization will need. Perhaps even the IP address block is 
> oversubscribed,
> so that there are more potential switch ports than IP 
> addresses, and you
> just hope that not everyone connects at once (or if they do, 
> they don't
> get an IP address).
> 
> So we're going to allow basically random assignment of IP addresses
> within the IP address block to each of the four organizations.
> 
> But you still want to keep the organizations separate, as in, 
> they don't 
> see each other's traffic. They
> don't get bothered with each other's ARPs and so forth. And you don't 
> want to change the IP nodes
> (endnodes or routers)
> to know anything funny is going on.
> 
> So you use VLANs to separate the communities. A particular port
> on a switch/bridge in a L2 cloud
> is configured as to what community the attached endnode belongs to, by
> having it configured with a VLAN ID.
> 
> But you'd like all the IP nodes in the cloud, even though in different
> communities, to share the same IP router, also possibly other 
> services such
> as the DHCP server. And we don't want the IP router
> to have to know about the different communities. The IP router just
> thinks the cloud is one big happy can't-we-all-just-get-along 
> IP subnet.
> 
> But the bridges inside the L2 cloud have to somehow do two things:
> a) enforce some sort of separation between the communities, and
> b) allow them all to talk to the IP router.
> 
> So this is how they are doing this. Assign each of the 4 
> communities a 
> VLAN ID, say
> VLANs A, B, C, and D. Now what VLAN ID should the IP router 
> belong to? 
> You want it
> to be able to talk to nodes in any of the VLANs {A, B, C, or D}.
> 
> The way this is done is to declare a VLAN group known as a 
> FID (filtering
> ID for those that want to see an acronym expanded -- personally, the
> expansion of that acronym didn't help me understand what a FID is).
> So a FID is a group of VLAN IDs, say Z, A, B, C, D, with one 
> of the IDs,
> say Z, being the "primary". The IP router that serves all the 
> communities
> (in this case, A, B, C, D) will be assigned to the "primary" VLAN, in 
> this case, Z. Endnodes
> will be in one of {A, B, C, D}. IP routers serving these 
> communities will
> be assigned VLAN Z.
> 
> To clarify, if there are n communities, there will be n+1 VLANs, one
> for each of the communities, and one for the IP router(s) serving
> the communities in that IP subnet.
> 
> Switches are configured to know that {Z, A, B, C, D} is a special 
> grouping of VLANs, such
> that something transmitted from a VLAN Z port goes to all 
> ports in the 
> group, i.e., all ports in
> Z, A, B, C, and D. However, something transmitted from a VLAN A port 
> goes only to ports in
> VLAN A and VLAN Z. Something from VLAN B goes to ports in 
> VLAN B and VLAN Z.
> 
> *************
> Now a little quiz...
> 
> For those of you that are following-- and in case I actually have 
> captured the concept,
> let me ask a question.
> 
> How do two endnodes in the IP subnet talk to each other?
> The first case is two endnodes in VLAN A, say S and D.
> S, observing that D is in the same subnet, will ARP. Since D 
> is in the 
> same VLAN, it will receive
> the ARP request, and respond. Everything works fine.
> 
> But how do two endnodes in different VLANs, but in the same 
> IP address 
> block communicate? S will
> ARP, like before, because IP nodes are (blissfully) unaware of VLANs. 
> The answer people tell me
> is "in that case communication between S and D has to go 
> through the IP 
> router". OK. So how would
> that work? The
> answer I get is "the switch does proxy ARP on behalf of the 
> IP router". 
> I don't understand that. How does the
> switch know *when* to proxy ARP? The switch doesn't necessarily know 
> which VLAN node D is in.
> 
> *******************
> Well, bravely continuing on...
> 
> 
> 
> Endnode MAC address learning is done by VLAN as before. If a frame is
> received by bridge b on
> port p, with source S, from VLAN A, then bridge b remembers that {S, 
> VLAN A} is on port p.
> 
> Furthermore, a Z-tagged unicast frame is checked against the learning
> tables of Z, A, B, C, D, to determine where to forward it. So 
> if bridge 
> b receives
> a frame with
> destination=D, bridge b checks for D in any of the VLANs Z, 
> A, B, C, D, and
> forwards accordingly.
> 
> 
> 
> &&&&&&&&&&&&&&&&&
> So, that's how bridges work (I hope). So how would we support this 
> functionality in
> RBridges?
> 
> As it turns out, despite the complexity of the concept,
> it will not be that difficult to support this with RBridges, and
> even in a somewhat more optimal way, since RBridges can do multicast
> filtering within the core rather that just at the final port.
> 
> RBridges, like bridges, would have to be configured with FIDs, i.e., 
> VLAN groupings as described above.
> 
> Let's call a FID by the name of the "primary" VLAN; in our example, Z.
> 
> RB1 would be configured that FID Z = {Z, A,B,C, D}, where Z, 
> A, B, C, D are
> all 12-bit VLAN IDs.
> 
> To avoid requiring configuration of all RBridges with these FIDs, and
> still allow multicast filtering, I propose we have RBridges advertise,
> in their IS-IS core instance, FIDs that they are configured 
> with. So at 
> I think VLANs Z, A, B, C, and D form a FID with primary=Z" 
> and the other 
> RBridges will, I guess
> believe him.
> 
> What to do if RB1 says that FID Z = {Z, A, B, C, D}, and RB2 says that
> FID Z = {Z,A,D, F} is an interesting question, but at least there is
> enough information to log an error. Lot of possibilities for 
> overlapping 
> FIDs, inconsistent FIDs, etc.
> I assume all those will be errors. If there is a FID called "Z", then 
> everyone better agree about what the
> VLAN membership of Z is, and none of the VLANs in Z better be in any 
> other FIDs.
> 
> Once there is a FID of {Z, A, B, C, D}, there will no longer be
> a VLAN-A instance of IS-IS, or a VLAN-B instance of IS-IS, or C or D. 
> Instead
> the Z-instance will announce VLANs A, B, C, and D membership 
> as well as 
> VLAN Z membership.
> 
> The Z-instance of IS-IS will specify which MAC addresses are 
> in which VLAN.
> So, RB1 might announce, in the Z-instance, {Z; a, b, q, d, f},
> {A; r, n, j}, {B; c, p, o, h}, {C; m,l}, {D;g, x, k}. The lower case 
> letters are endnode MAC
> addresses. The upper case letters are 12-bit VLAN IDs.
> 
> Multicast packets marked as VLAN Z (inner VLAN) will be sent to
> all links in Z, A, B, C, or D. So the LSPs in the VLAN Z instance of 
> IS-IS will
> be delivered to all RBridges in Z, A, B, C, and D.
> 
> That way RBridges with ports in Z, A, B, C, and/or D will 
> learn all the 
> endnode addresses
> in any of those VLANs (all the ports in FID Z).
> 
> If ingress RB1 is attached to Z, and receives a packet with
> destination D tagged as Z, RB1 checks for D in VLANs A, B, C, 
> D, and Z 
> 's endnode
> membership. As soon as RB1 finds it, let's say, as reported by
> RB2 as being in VLAN D, RB1 tunnels the packet in TRILL to RB2,
> leaving the tag as Z. When RB2 decapsulates the packet, I 
> assume it will
> rewrite the inner VLAN tag from "Z" to "D".
> 
> 
> &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
> Conclusions:
> 
> The changes to the spec:
> 
> a) announce in the core instance, FIDs as a set of VLANs, the
> first listed being the "primary".
> 
> b) multicast forwarding: if the packet is (inner) tagged with
> the VLAN ID of a primary VLAN in a FID, forward the packet on
> all in the links in the selected tree that lead to any of the VLANs
> in the FID. If the packet is tagged with the VLAN ID of a non-primary
> VLAN in a FID, forward the packet on all the ports that reach
> links in either that VLAN or in the primary VLAN for that FID.
> 
> c) when ingressing a packet with destination D, tagged with a
> VLAN tag of a primary VLAN in a FID, check the endnode information
> for all the VLANs in the FID to determine the egress RBridge.
> 
> d) when ingressing a packet with destination D, tagged with
> a VLAN tag of a secondary VLAN in a FID, check the endnode
> information for exactly two VLANs; the one the packet is
> currently tagged with, and the primary VLAN for that FID, to
> determine the egress RBridge.
> 
> e) if two RBridges, in the core instance, report inconsistent FID 
> membership, what should we do?
> 
> (Note: There was a fortune cookie program in Unix, one of the 
> fortunes 
> being "Never check for an error
> condition that you don't know how to handle". For the record--I think 
> that's a cute joke but really bad policy.).
> 
> Radia
> 
> 
>

---

From: Radia.Perlman at sun.com (Radia Perlman)
Date: Mon, 02 Apr 2007 13:10:04 -0700

My understanding is that shared VLANs assume
IP routers are not aware of VLANs.

My understanding is that the way it would work if the IP router R *was* 
aware of
VLANs, is that R would treat the link as 4 separate links (by using 
VLANs), and then the IP addresses
on the 4 different links (VLANs A, B, C, and D) would have to have 
different IP prefixes.

So therefore, I am assuming that the IP router is not supposed to know 
that the link is subdivided into VLANs.

So my understanding of the problem in this case is:

a) single IP address block shared by different communities
b) desire to isolate these communities from layer 2 broadcast traffic 
from each other
c) desire to share some services (like DHCP and IP routers), without 
those services being aware
that the communities are separate

As for my question about proxy ARP, to further add
to the confusion -- some people are assuming the bridge/RBridge will be 
doing
the proxy ARP, and some others are assuming the router is.

And actually, after talking offline with Joel Halpern, I'll send another 
note with two different proposals
(one just a cleanup of what I originally wrote, and the other a 
different approach) for solving
that vaguely stated problem.

Radia

---

From: caitlinb at broadcom.com (Caitlin Bestler)
Date: Mon, 2 Apr 2007 13:11:05 -0700

it...
> 
> 	You believe that the problem that shared VLAN learning
> is used to solve is conservation of IP addresses used in a
> single IP subnet, and that it is necessary to use shared VLAN
> learning to keep routers (in the subnet) from having to
> re-learn MAC+IP associations across a set of VLANs in that
> same IP subnet, is that correct?
> 

It's a more fundamental problem that re-learning, and the goal
could more properly be stated as avoiding the neccesity of 
creating distinct subnets. Sharing the IP subnet does conserve
IP Addresses, but more importantly it conserves the effort of
dividing the IP addresses up into VLAN specific portions, and
of updating DNS entries when a host migrates to a new VLAN, etc.

There was actually a very good summary of the usage of such
a feature on a BSD list in 2005:

http://lists.freebsd.org/pipermail/freebsd-net/2005-November/009064.html

Different switches having different VID/FID mappings is confusing,
but ultimately it isn't that bad, especially since global MAC addresses
do tend to be unique within a campus even when they are being assigned
by decidedly non-global techniques.

But the associated ARP Proxy has a totally different problem.
Specifically, the scenario is where the *same* IP subnet is
used for two or more VLANs. Or to be more precise, what is
to *appear* to be one IP subnet to the outside world is
actually two or more VLAN divided subnets, and the assignment
of specific IP addresses to specific VLANs is dynamically
learned rather than administered.

So if RBridges scope the MAC/IP learning to a VID
specific instance then two RBridges can have
contradictory information about the meaning of
an IP address.

In fact, a single RBridge could have contradictory
information in two VID-specific instances about the
same IP address.

This creates a severe problem for the router (existing
router, mind you) that is the ingress point to this
shared IP subnet. When it has a received IP datagram
to x.y.z.1, and no entry in its ARP table, it will
have no idea which VLAN the datagram belongs to.
Therefore it will ARP on all VLANs to see where
it goes, and it expects to get at most one reply
because an IP address is only supposed to be
assigned to a single host. Not that the router
will get upset or have a fault. It will more
likely assume to all replies are redundant and
therefore it can always just use the most recent
one without bothering to remember any prior ARP
responses.

So, when a host migrates from VLAN A to VLAN B,
we have a problem if the RBridge on VLAN B does
not share its discovery with the RBridge on VLAN
A (and the two are part of a group / share the
same IP subnet). If the discovery is shared, then
the RBridge on VLAN A will delete its entry for
the IP address, and when the Router next ARPs
it will only get a response from VLAN B.

So while the groups that Radia describes will
solve the problem, the fundamental justification
for the solution arises when the semantics of
an IP subnet span multiple VIDs. When the
IP Address retained for Proxy ARP purposes
has a wider scope that the VLAN dissemination
of discovery cannot be confined to the VLAN.

And while I could see actually making the determination
based on IP Addressing, I don't see any real advantage
to doing so.

While we've been refining the examples, the underlying
problem remains as originally stated. Since RBridges
provide a distributed Proxy ARP service they simply
must avoid having contradictory information held 
by different RBridges (they can and should have
different subsets of the entire picutre, but no
RBridge should ever have data that contradicts the
data that other RBridgess have).

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Mon, 2 Apr 2007 15:50:05 -0500

Radia,

	There is an interesting problem that might arise out of
a router receiving VLAN tagged frames without being aware that
they are tagged.  

	A router is a MAC termination point - i.e. - it is an 
end-station at the MAC layer.  But it is also a forwarding
device.  Hence the slack that might apply to a host receiving
a MAC frame with "stuff" in the header it doesn't understand,
shouldn't be allowed for a router.

	Also, many routers do understand that it is possible to
have the same IP subnet connected to multiple logical interfaces.
That was where the original "Bridging Router" (or BRouter) came
from.

--
Eric Gray
Principal Engineer
Ericsson  

> From: Radia Perlman [mailto:Radia.Perlman at sun.com] 
> Sent: Monday, April 02, 2007 4:10 PM
> To: Eric Gray (LO/EUS)
> Subject: Re: [rbridge] Shared VLAN learning: What is it and 
> why should we care?
> Importance: High
>

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Mon, 2 Apr 2007 14:05:47 -0700

Eric,

On
> Behalf Of Eric Gray (LO/EUS)
> Sent: Monday, April 02, 2007 1:50 PM
> To: Radia Perlman

This is not the case, unless there is a misconfiguration. A Router has a
concept of an interface (e.g. eth0, the Ethernet port) and of
sub-interfaces (e.g. eth0/3). VLANs and subnets are associated to
sub-interfaces. When a frame arrive the VID (VLAN ID), either explicit
or implicit, is used to select a sub-interface. The sub-interface has an
IP address and a netmask.

-- Silvano

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Mon, 2 Apr 2007 16:08:23 -0500

Radia,

	WRT your quiz question:

	To expand on Sanjay's earlier reply (assuming I've understood 
your questions correctly), you've assumed an essentially broken IP
subnet configuration.  That doesn't mean that it wouldn't work with
today's devices, but it does mean that it is not working the way I
believe you have described it.

	Sanjay is correct about ARP as a function of the IP router.  A
"smart bridge" may incorporate enough of the routing function to help
out in this scenario, but is not a standards defined implementation
if it does.  There is a fairly long list of caveats that must apply 
in that case, and this is not an area we want to explore - unless it
turns out to be the case that someone does this incorrectly and we
find that we need to be compatible with the incorrect implementation.

	In one case, there may be a VLAN aware bridge between the IP
router and the 4 VLANs you describe.  In implementations, this VLAN
aware bridge may actually be co-located with the IP router.  In this
case, the VLAN aware bridge may very well forward un-tagged frames to
and from the (potentially co-located) IP router, across one (possibly
logical) link, allowing the router to enjoy simplified ARP learning.
This VLAN aware bridge then provides the VLAN translation function to
allow proper forwarding of data within the subnet.

	Note that - in this case - the VLAN aware bridge is translating
untagged frames into tagged frames (and vice versa), but is not doing
the similar translation (one tag to another) between distinct tagged
VLANs.  That function must be done by a router - hence the router is
supposed to handle ARP responses and associate IP-to-MAC so that it is 
in the forwarding path (much as Sanjay described).

	Because you're assuming over-lapping assignments of IP
addresses,
this scenario causes problems for a standards defined IP router because 
it will see all of the subnet-local IP destinations as being part of a
single IP subnet, and will - most likely - try to force the senders in 
the IP sub-network to send directly to destination MAC addresses - via
ARP responses (assuming single subnet) and IP re-directs.

	As you probably realize, that situation would be pathological.

	This situation arises in real networks, and has been addressed
in real implementations.  One approach blurs the distinction between 
the VLAN aware bridging and standard routing functions.  

	For the co-located case, this is not hard: the router sees more
than one VLAN on a single physical interface, but is configured (or
learns) that these multiple logical interfaces are all part of the same
IP subnet.  In this case, the router is required to bridge between the
multiple logical interfaces that are part of a single logical IP subnet.

	An implementation may instantiate a VLAN-aware bridging instance

to handle this role.  In this case, the boundary between bridging and
routing becomes blurred because the same physical device is involved
in both - making the handling of ARP and appropriate VLAN forwarding 
work because the router can carefully choose which role it assumes in
dealing with ARP and forwarding of traffic.  If this takes place in a
single device, it is possible to avoid many of the caveats that would
apply if this particular form of cheating was distributed.

	In the non-colocated case, a bit more work is required to make
the scenario functional and a number of caveats cannot be completely
avoided.  For example, if there is exactly one VLAN aware bridge in
the local IP-subnet, it is possible for that bridge to "proxy" ARP
responses (as if it were a router) and forward frames between VLANs.
If there are multiple VLAN aware bridges, then it should be possible 
to configure them so that only one of them performs this function.
A more complex configuration is possible, but such a configuration is
workable only if there is no potential forwarding path (including any
possible failure mode) which results in multiple VLAN ID translations
- and forwarding - between multiple VLAN aware bridges, in a loop.

	For RBridges, the situation we need to avoid is as shown below:

                 Host Group-P
                    A
                    |
             _____VLAN-A______
            |                 |
            V                 V
   R <---> VAB <--VLAN-B--> RBridge CRED/Campus <--VLAN-B--> Host
                                                            Group Q

	If both the RBridges and the VLAN aware Bridge (VAB) are doing
Proxy ARP - and VLAN ID translation (cross-VLAN forwarding) - it is 
possible for frames to be forwarded consistently back and forth between
the RBridge CRED/Campus and VAB.  Moreover, it is possible for a host
in group P in this diagram, to get a copy of each frame each time it
goes around.

	Still another alternative is that IP addresses are not assigned
as you suggest, and VLAN tags are used simply to share physical link
resources - while maintaining IP address space separation.  Many of
the router vendors today (I believe) would claim that the alternative
is pathological...

--
Eric Gray
Principal Engineer
Ericsson

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Mon, 2 Apr 2007 16:15:35 -0500

Silvano,

	Uhhhh, yes, but that makes the router aware of VLANs - doesn't
it?

	Getting back to Radia's statement (below) - "So therefore, I
(Radia)
am assuming that the IP router is not supposed to know that the link is 
subdivided into VLANs."

	I think you need to review the order of the dialog.  I was
pointing
out that Radia's assumption of router ignorance of VIDs would be a
problem.
You apparently would argue that routers are not ignorant of VIDs.

	So, unless I am misunderstanding you, I believe we are in
agreement.

--
Eric Gray
Principal Engineer
Ericsson  

> From: Silvano Gai [mailto:sgai at nuovasystems.com] 
> Sent: Monday, April 02, 2007 5:06 PM
> To: Eric Gray (LO/EUS); Radia Perlman
> Subject: RE: [rbridge] Shared VLAN learning: What is it and 
> why should wecare?
> Importance: High
> 
> Eric,
> 
> why should
> > > >(in the subnet) from having to re-learn MAC+IP 
> associations across
> a
> > > >>FID Z = {Z,A,D, F} is an interesting question, but at 
> least there
> is

---

From: caitlinb at broadcom.com (Caitlin Bestler)
Date: Mon, 2 Apr 2007 14:37:29 -0700

rbridge-bounces at postel.org wrote:
>> On Behalf Of Eric Gray (LO/EUS) Sent: Monday, April 02, 2007 1:50 PM
>> To: Radia Perlman
>> Subject: Re: [rbridge] Shared VLAN learning: What is it and why
>> should wecare? 
>> 
>> Radia,
>> 
>> 	There is an interesting problem that might arise out of a router
>> receiving VLAN tagged frames without being aware that they are
>> tagged. 
>> 
> 

If you review the email I cited previously, and then do some googling,
you'll see that assigning each VLAN a recognizable IP subnet is merely
the predominant technique -- it is not the only one, and it is not a
requirement.

Do you agree that when a router connects to multiple VLANs that share
an IP subnet that having the RBridge generate a Proxied ARP response
to the *same* IP address on each VLAN is a problem that needs to
be solved?

How do you prevent the incorrect Proxied ARP response without sharing
the endnode discovery over the truly relevant span, rather than just
within the VLAN?

This is a more pressing issue than just Shared vs. Independent Learning.
Mandating that all RBridges use Independent Learning is undesirable to
the extent that specifications should be minimalistic and impose as
few constraints as possible. But realistically I doubt that Shared
Learning is really desirable for RBridges.

The shared IP subnet issue, however, has nothing to do with RBridges.
Forbidding sharing of IP subnets across VLANs would be considerably
more drastic, and presumably outside the WG's scope. For one thing
it would directly contradict RFC 3069:

   The VLAN [802.1Q] aggregation technique described in this document
   provides a mechanism by which hosts that reside within the same
   physical switched infrastructure, but separate virtual broadcast
   domains, may be addressed from the same IPv4 subnet and may share a
   common default gateway IPv4 address.

...

   Essentially, what occurs is that each sub-VLAN (customer) remains
   within a separate broadcast domain.  One or more sub-VLANs belong to
   a super-VLAN, and utilize the default gateway IP address of the
   super-VLAN.  Hosts within the sub-VLANs are numbered out of IP
   subnets associated with the super-VLAN, and their IP subnet masking
   information reflects that of the super-VLAN subnet.

   If desired, the super-VLAN router performs functions similar to Proxy
   ARP to enable communication between hosts that are members of
   different sub-VLANs.

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Mon, 2 Apr 2007 14:44:41 -0700

Eric,

Please review IEEE 802.1Q-2005 B.1.3 Asymmetric VLANs.
In figure B-4 assume a router instead of the Server.
As you can see the link between the router and the Bridge is untagged.
Being the link untagged the router will have a single sub-interface
associated to a single IP subnet.

Nonetheless it will be on the Red, Blue and Purple Asymmetric (Private)
VLANs. The frames received by the router will be legal IP frames.

I don't know if we are in agreement, but this is the most pertinent
example I am able to come out with.

-- Silvano

On
> Sent: Monday, April 02, 2007 2:08 PM
> To: Radia Perlman
and
advertise,

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Mon, 2 Apr 2007 16:51:50 -0500

Silvano,

	Possibly this was meant as a reply to a different message?

	If you look at the text below, one of my early paragraphs
(the one that starts "In one case, there may be a VLAN aware bridge 
between the IP router and the 4 VLANs ...") is nearly identical to
the illustration you describe.  So, I think we can assume we're at
least talking about the same thing.

	Which leaves me wondering what your point is...

Thanks!

--
Eric Gray
Principal Engineer
Ericsson  

> Sent: Monday, April 02, 2007 5:45 PM
> > > is configured as to what community the attached endnode 
> belongs to,
> by
> > > Let's call a FID by the name of the "primary" VLAN; in 
> our example,
> Z.
> > >
> least there is

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Mon, 2 Apr 2007 17:15:09 -0500

Caitlin,

	I'm not sure who you're addressing this to, but I'll take a
crack at answering it.  Please see below...

Thanks!

--
Eric Gray
Principal Engineer
Ericsson  

> From: Caitlin Bestler [mailto:caitlinb at broadcom.com] 
> Sent: Monday, April 02, 2007 5:37 PM
> To: Silvano Gai; Eric Gray (LO/EUS); Radia Perlman
> [mailto:rbridge-bounces at postel.org]

I agree.  However, solving it may be simply requiring that
implementations
are configurable to avoid the pathological consequences of this
situation.

Since this would be the case today with existing VLAN aware devices, I
am
not sure what we (TRILL) needs to say or whether or not it even makes
any
sense for us to say anything.

It is usually not necessary to specify that implementers should not
build
broken implementations.

> the endnode discovery over the truly relevant span, rather than just

So, here's my take:

o	If SVL is in use, then it is usually necessary that all of the
bridges
	have a consistent view of which VLANS are shared (this is
independent 
	of a specific FID <-> VID mapping except that the same VIDs
SHOULD be
	mapped to a single locally determined FID in all bridges in a
given
	deployment).
o	This would typically be a matter of configuration, although it
is 
	possible that configration tools may be used to help ensure that
a
	consistent picture is "shared" among bridges.
o	Existence of VLANs in a deployment typically means configuration
is
	required to make the VLANs work (hence this is not strictly a
_new_ 
	requirement to steer us away from the plug-and-play objective).
o	Whether SVL, or IVL, is used - if IP-MAC binding information is
only
	distributed where appropriate (based on VID) at each RBridge,
then
	it is difficult for me to see how incorrect proxied ARP
responses 
	would occur.
o	Even if all IP-MAC binding information was distributed to all
RBridges,
	as long as this information is scoped by the designated RBridge
in any
	ARP responses it might provide (by relevant VID in the request),
then
	it is still difficult to see how incorrect proxied ARP responses

	might occur.

> Learning is really desirable for RBridges.

Okay.  Do I get to say "I told you so" when you get aroud to eating that
opinion?  :-)

>    within a separate broadcast domain.  One or more sub-VLANs belong
>    to a super-VLAN, and utilize the default gateway IP address of the

I think we should avoid the "super-VLAN" concept.  What you seem to
be talking about is IP subnets, rather than VLAN aggregation.

> 
>    If desired, the super-VLAN router performs functions similar to
>    Proxy ARP to enable communication between hosts that are members
>    of different sub-VLANs.
> 
>

---

From: jrrivers at nuovasystems.com (J. R. Rivers)
Date: Mon, 2 Apr 2007 15:51:42 -0700

These are issues that are commonly dealt with today when routers (either
stand alone or "merged L3 switches") face this situation today.  I
suggest focusing TRILL on retaining compatibility with today's
expectations rather than altering them in any way.

JR

> [mailto:rbridge-bounces at postel.org] On Behalf Of Eric Gray (LO/EUS)
> associations across a
> > belongs to, by

---

From: Radia.Perlman at sun.com (Radia Perlman)
Date: Mon, 02 Apr 2007 16:00:20 -0700

Based on the thread, and some off-list communication with Joel Halpern 
and others, I now
would like to propose two concrete solutions.

I'd like people to make comments like
. "either of them is fine, but I prefer this one because..."
. "neither of them solve the problem. The problem is ..."
. "here is a better option .. xxxxxxxxx"

Option 1: Making groupings known to all the RBridges (slightly revised 
from my original proposal)
--------------------------------------------------------------------------------------------------

a) Make a group of VLANs {Z, A, B, C, D}, where VLAN Z is the "primary" and
{A, B, C, D} are "secondaries".  All RBridges learn about this grouping 
because
at least one RBridge RB1 has been configured to know these 5 VLANs 
belong in a group,
and RB1 announces this grouping in its core IS-IS instance.

A frame tagged with VLAN Z (say an ARP request
from the IP router) goes to all the links in all the VLANs in the group 
(i.e., Z, A, B, C, and D). While
in the core, the inner VLAN tag is Z. When an RBridge decapsulates onto 
a VLAN A link, it changes the
VLAN tag to A in order to forward it (or removes the VLAN tag).

RBridges in the core do optimal filtering of VLAN-tagged packets,
so that something tagged VLAN D goes to D and Z, something marked Z goes 
to Z, A, B, C, and D, and
something marked F (not in a group) goes just to F. And the distribution 
tree gets filtered as early
as possible, by some core RBridge that notices no downstream link leads 
to any of the VLANs in the
set it is supposed to be delivering to.

There will still be separate per-VLAN instances of  Z, A, B, C, and D. 
These are not combined.
RBridges that only claim to be attached to VLAN Z, will also participate 
in the per-VLAN IS-IS
instances of A, B, C, and D, in that they will recieve and store the 
link state announcements about endnodes
in those VLANs.

If two different RBridges announce different groupings, say RB1 
announces {Z, A, B, C, D} and
RB2 announces {Z, A, E}, then we log an error, but core RBridges
treat the group as the set union, i.e. {Z, A, B, C, D, E}, for the 
purpose of filtering multicasts tagged with
any of those VLAN tags.

Option 2: Groupings only known to RBridges at the edge involved in these 
groupings
-------------------------------------------------------------------------------------

b) Only RBridges associated with these groupings know anything about the 
grouping. So going with
our original example, an IP router is shared by VLANs A, B, C, and D, 
and is assigned to VLAN Z.
The IP router is not aware of any of this, and assumes everything is one 
IP subnet. RBridges
attached to IP nodes in any of the VLANs {Z, A, B, C, D} *are* aware of 
this grouping.

Suppose the IP router is
attached to RBridge RB1. The IP router does an ARP request. RB1 knows 
that something from
the IP router is supposed to go to
VLANs Z, A, B, C, and D, so RB1 duplicates the packet (in this case, the 
ARP request),
creating 5 packets, and transmits one
into the cloud tagged as VLAN Z, one as VLAN A, one as VLAN B, one as 
VLAN C, one as VLAN D.
In the IP router issues a unicast packet to MAC address M, RB1 looks in 
its endnode tables
for VLAN Z, VLAN A, VLAN B, VLAN C, and VLAN D. If RB1 finds M, say in 
VLAN B,
RB1 tags the packet as "VLAN B", and sends it to the egress RBridge that 
announced attachment to M
in VLAN B. I assume that RB1 stops when it finds M, and if M is in 
multiple of these tables, too bad.
RB1 sends it to the first one it finds.

For an RBridge RB2 attached to a "secondary VLAN", say C, the same thing 
happens.
Multicasts from C get duplicated so RB2 sends two copies, one marked 
"VLAN C", and one marked "VLAN Z".
For unicast, RB2 sees if it can find the MAC address in its C table or Z 
table, and if it does, unicasts it
to the egress RBridge (marking the VLAN tag according to the table in 
which it found the destination,
or leaving it as C; I think either one will work).

But if RB2 does *not* find the destination in any of the endnode tables, 
RB2 creates two copies of the packet, one
marked VLAN Z, one marked VLAN B, and multicasts each one.

********************************************************
Pros and cons of these two proposals

Proposal a) is more optimal since packets to the group do not need to be 
duplicated,
and furthermore get filtered in the core. However, it requires adding an 
announcement
into the core IS-IS instance of what the VLAN groupings are, and some 
thought as
to what to do if different RBridges announce different groupings.

Proposal b) adds less complexity to the basic RBridge spec. Since no 
sharing of
grouping information is done, misconfiguration will not be detectable.

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Tue, 3 Apr 2007 06:22:45 -0500

JR,

	I agree.

--
Eric Gray
Principal Engineer
Ericsson  

> From: J. R. Rivers [mailto:jrrivers at nuovasystems.com] 
> Sent: Monday, April 02, 2007 6:52 PM
> To: Eric Gray (LO/EUS); Radia Perlman; rbridge at postel.org

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Tue, 3 Apr 2007 08:07:42 -0500

Radia,

	I imagine there are many more ways to answer this question
than the three alternatives you've offered as examples.

	I would answer this way:

.	neither approach is required to solve the problem, hence I
	suggest we concentrate on not breaking solutions that work
	now;

	And:

.	Here is a better option - we specify that an RBridge may be
	configured to provide an inter-VLAN proxy ARP service in a
	manner that is compatible with similar services offered at
	present and the mechanisms and potential protocols needed
	to accomplish this are out of scope for TRILL.

--
Eric Gray
Principal Engineer
Ericsson  

> Sent: Monday, April 02, 2007 7:00 PM
> Subject: [rbridge] Two "shared VLAN" alternative proposals
>

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Tue, 3 Apr 2007 08:05:36 -0700

I completely agree

-- Silvano

On
> Sent: Tuesday, April 03, 2007 4:23 AM
> To: J. R. Rivers; Radia Perlman; rbridge at postel.org
> shouldwecare?
is used

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Tue, 3 Apr 2007 08:16:03 -0700

Eric,

I will go one step further and say that the basic TRILL protocol should
not specify an ARP/ND proxy function.

As the discussion as pointed out, ARP/ND proxy is more a product feature
that depends on how the box are packaged. IMHO RBridges will be packaged
with additional router, NAT, firewall functionalities. If and where
proxy ARP will be implemented will vary.

After the last call on the base protocol, we can always resume the work
and deal with Proxy ARP/ND in a separate document.

-- Silvano

On
> Sent: Tuesday, April 03, 2007 6:08 AM
> To: Radia Perlman; rbridge at postel.org
> Subject: Re: [rbridge] Two "shared VLAN" alternative proposals
> 
> Radia,
>

---

From: ddutt at cisco.com (Dinesh G Dutt)
Date: Tue, 03 Apr 2007 08:27:36 -0700

I can't agree more. The entire ARP/ND optimization defocusses from the 
main strengths of TRILL: equal cost multipathing and shortest path 
forwarding at L2. I had suggested that we move the ARP/ND optimization 
section out of the base protocol spec into a separate document that can 
be addressed separately. I'd like to reiterate that suggestion.

BTW, Private VLANs was *only an* example of why shared VLAN learning is 
needed. During the meeting, we (Silvano and I) raised a list of open 
items and SVL was one of them. People questioned the need for this 
functionality and Ali Sajassi and I stated one example of its use. 
Focussing on how it works and attempting to solve what Private VLAN does 
outside of supporting SVL is an overkill, at least for the initial 
versions.

Dinesh
J. R. Rivers wrote:
>>> b) desire to isolate these communities from layer 2 
>>>       
>> broadcast traffic 
>>     
>>> from each other
>>> c) desire to share some services (like DHCP and IP 
>>>       
>> routers), without 
>>     
>>>> to solve is conservation of IP addresses used in a single 
>>>>         
>> IP subnet, 
>>     
>>>> and that it is necessary to use shared VLAN learning to 
>>>>         
>> keep routers 
>>     
>>>>> 100 addresses, and there are 4 organizations. One 
>>>>>           
>> strategy might be
>>     
>>>>> wind up having separate IP subnets (if the addresses can 
>>>>>           
>> be assigned
>>     
>>>>> So we're going to allow basically random assignment of IP 
>>>>>           
>> addresses
>>     
>>>>> don't get bothered with each other's ARPs and so forth. And 
>>>>>           
>>> you don't 
>>>       
>>>>> But you'd like all the IP nodes in the cloud, even though 
>>>>>           
>>> in different
>>>       
>>>>> to have to know about the different communities. The IP 
>>>>>           
>> router just
>>     
>>>>> ID for those that want to see an acronym expanded -- 
>>>>>           
>> personally, the
>>     
>>>>> expansion of that acronym didn't help me understand what 
>>>>>           
>> a FID is).
>>     
>>>>> (in this case, A, B, C, D) will be assigned to the 
>>>>>           
>>> "primary" VLAN, in 
>>>       
>>>>> this case, Z. Endnodes
>>>>> To clarify, if there are n communities, there will be n+1 
>>>>>           
>> VLANs, one
>>     
>>>>> Z, A, B, C, and D. However, something transmitted from a 
>>>>>           
>>> VLAN A port 
>>>       
>>>>> goes only to ports in
>>>>> ARP, like before, because IP nodes are (blissfully) unaware 
>>>>>           
>>> of VLANs. 
>>>       
>>>>> switch know *when* to proxy ARP? The switch doesn't 
>>>>>           
>>> necessarily know 
>>>       
>>>>> which VLAN node D is in.
>>>>>
>>>>> *******************
>>>>> Endnode MAC address learning is done by VLAN as before. If 
>>>>>           
>>> a frame is
>>>       
>>>>> received by bridge b on
>>>>> port p, with source S, from VLAN A, then bridge b remembers 
>>>>>           
>>> that {S, 
>>>       
>>>>> VLAN A} is on port p.
>>>>>
>>>>> Furthermore, a Z-tagged unicast frame is checked against 
>>>>>           
>>> the learning
>>>       
>>>>> So, that's how bridges work (I hope). So how would we 
>>>>>           
>> support this 
>>     
>>>>> functionality in
>>>>> RBridges?
>>>>>
>>>>> even in a somewhat more optimal way, since RBridges can 
>>>>>           
>> do multicast
>>     
>>>>> RBridges, like bridges, would have to be configured with 
>>>>>           
>>> FIDs, i.e., 
>>>       
>>>>> Let's call a FID by the name of the "primary" VLAN; in our 
>>>>>           
>>> example, Z.
>>>       
>>>>> To avoid requiring configuration of all RBridges with these 
>>>>>           
>>> FIDs, and
>>>       
>>>>> What to do if RB1 says that FID Z = {Z, A, B, C, D}, and 
>>>>>           
>>> RB2 says that
>>>       
>>>>> I assume all those will be errors. If there is a FID called 
>>>>>           
>>> "Z", then 
>>>       
>>>>> VLAN membership of Z is, and none of the VLANs in Z better 
>>>>>           
>>> be in any 
>>>       
>>>>> other FIDs.
>>>>>
>>>>> a VLAN-A instance of IS-IS, or a VLAN-B instance of IS-IS, 
>>>>>           
>>> or C or D. 
>>>       
>>>>> Instead
>>>>> {A; r, n, j}, {B; c, p, o, h}, {C; m,l}, {D;g, x, k}. The 
>>>>>           
>>> lower case 
>>>       
>>>>> letters are endnode MAC
>>>>> all links in Z, A, B, C, or D. So the LSPs in the VLAN Z 
>>>>>           
>>> instance of 
>>>       
>>>>> IS-IS will
>>>>> all in the links in the selected tree that lead to any of 
>>>>>           
>> the VLANs
>>     
>>>>> in the FID. If the packet is tagged with the VLAN ID of a 
>>>>>           
>>> non-primary
>>>       
>>>>> condition that you don't know how to handle". For the 
>>>>>           
>>> record--I think 
>>>       

-- 
We make our world significant by the courage of our questions and by 
the depth of our answers.                               - Carl Sagan

---

From: caitlinb at broadcom.com (Caitlin Bestler)
Date: Tue, 3 Apr 2007 08:46:13 -0700

> 	I imagine there are many more ways to answer this 
> question than the three alternatives you've offered as examples.
> 
> 	I would answer this way:
> 

I think this may be a good approach. The only RBridge related
problem with inter-VLAN quasi-routing that has been identified
is an artifact of the ARP Proxy functionality. It makes sense
to fix it there.

But I do agree with Radia that there are two essential ways
to solve this:

	Extend the information collected during RBridge discovery
	so that this problem will be taken care of automatically.

	or, require the RBridges that have this special problem
	to solve it amongs themselves and leave everyone else
	out of it.

With the first method you retain zero configuration for
this feature by advertising VLAN groupings that require
what would otherwise be excessive distribution of endnode
discovery announcements. What Radia proposed works, but
alternate approaches are possible if there is something
inherently wrong about explicitly exporing FID info.

With the second method you would be explicitly configuring
each RBridge within the group with the additional export
rules. Zero configuration is lost for those RBridges, but
kept simpler for most RBridges.

But unless I'm missing something, the only special action
required is to ensure that when a group of VLANs share an
IP subnet that all Proxy ARP tables that reference the 
shared IP subnet will be told of new discoveries.

---

From: jmh at joelhalpern.com (Joel M. Halpern)
Date: Tue, 03 Apr 2007 13:04:15 -0400

I think we can take the description a step further.

As long as we do not insist that all special cases must be solved 
optimally, it seems that this particular case, as described, can be 
solved with no modification to the protocol just by special behavior 
of the RBridge at the router.  Given that it is a special case, and 
that it can be solved externally to the protocol, it would seem to 
make sense for the protocol not to solve this problem.

We do not normally specify what additional features, outside of the 
spec, an implementation may or may not perform.  Hence, the most we 
might consider doing for this is an infromational appendix that 
describes how this particular case is solved.  (And remember, we can 
not describe how to solve all the possible special cases.)

Yours,
Joel M. Halpern

---

From: caitlinb at broadcom.com (Caitlin Bestler)
Date: Tue, 3 Apr 2007 10:28:06 -0700

> [mailto:rbridge-bounces at postel.org] On Behalf Of Joel M. Halpern
> Sent: Tuesday, April 03, 2007 10:04 AM
> As long as we do not insist that all special cases must be 
> solved optimally, it seems that this particular case, as 
> described, can be solved with no modification to the protocol 
> just by special behavior of the RBridge at the router.  Given 
> that it is a special case, and that it can be solved 
> externally to the protocol, it would seem to make sense for 
> the protocol not to solve this problem.
> 
> We do not normally specify what additional features, outside 
> of the spec, an implementation may or may not perform.  
> Hence, the most we might consider doing for this is an 
> infromational appendix that describes how this particular 
> case is solved.  (And remember, we can not describe how to 
> solve all the possible special cases.)
> 

What started this whole line of discussion was that the protocol
specification at the time explicitly stated that end node discovery
was only shared with the VLAN. That leads to problems when RBridges
that are not part of VLAN A need to know of a discovery in VLAN A
because it implies removal of conflicting key addresses (IP or MAC)
in their own data structures.

So, the specification could merely state that RBridges MAY
communicate endnode discovery with other RBridges for whatever
reasons. It could even cite shared IP subnets as an example
of a special feature that specific RBridges MAY chose to
support.

That might make sense if we adopt the "it's not a general problem
that should be handled by the default zereo configuration solution"
approach.

---

From: Radia.Perlman at sun.com (Radia Perlman)
Date: Tue, 03 Apr 2007 11:25:25 -0700

It that's all that's needed (letting RBridges in Z know about membership 
of A, B, C, and D), then
that would work by having RBridge RB1, which is only attached to, say, 
Z, advertise in its core IS-IS
instance that it actually is also attached to A, B, C, and D. That way 
RB1 will receive the endnode
advertisements of A, B, C, and D as well as Z.

Radia

---

From: caitlinb at broadcom.com (Caitlin Bestler)
Date: Tue, 3 Apr 2007 11:29:36 -0700

> Sent: Tuesday, April 03, 2007 11:25 AM
> To: Caitlin Bestler
> Cc: Joel M. Halpern; rbridge at postel.org
> It that's all that's needed (letting RBridges in Z know about 
> membership of A, B, C, and D), then that would work by having 
> RBridge RB1, which is only attached to, say, Z, advertise in 
> its core IS-IS instance that it actually is also attached to 
> A, B, C, and D. That way

If you are in {Z,A} you need to know about endnodes in B
because of the shared address space problem, but you do
not want B's traffic. It is *only* the endnode advertisements
that you want added.

---

From: jmh at joelhalpern.com (Joel M. Halpern)
Date: Tue, 03 Apr 2007 14:43:53 -0400

I don't follow the connection.

>in their own data structures.

Discover that MAC address X is on VLAN B should not always imply 
removal of MAC address X from VLAN A.  A device may use the same 
address on multiple VLANs.  I understand that there are cases where 
such sharing of information may be necessary.
But those cases have nothing to do with the situation of a router 
unknowingly serving several VLANs all within the same subnet.  In 
fact, as far as I can tell, that scenario is best dealt with if the 
routers MAC address is permitted to appear in several different VLANs 
simultaneously.

>of a special feature that specific RBridges MAY chose to

The problem of proper removal of stale information (either because of 
exclusive VLAN usage where the end station is not permitted to be on 
several VLANs, or because of physical movement, is generally best 
addressed by explicit and direct support, if it is desired.  Trying 
to have unspecified behavior produce consistent result just doesn't work.
However, it also does not seem to be related to the scenario as 
described in Radia's email and the IEEE spec.  There is no change in 
what VLAN can be used to reach any given MAC address in those situations.

>that should be handled by the default zereo configuration solution"

the grouping of VLANs (whether for an end-RBridge support or for 
Radia's proposal 1 advertisement, or any other approach) is clearly 
not going to supported by the zero-configuration case.  It can't be.

Yours,
Joel

---

From: Radia.Perlman at sun.com (Radia Perlman)
Date: Tue, 03 Apr 2007 11:45:23 -0700

Re: Caitlin's comment below

Really? The problem, as explained to me by some subset of the people, 
was that the IP router R
would be on the primary VLAN (in this case, Z), and so R needs to 
receive at least some broadcasts from
the secondaries -- the ARP/ND queries, for example.

It seemed to me that VLAN Z traffic (unicast or multicast) has to be 
legally accepted by all links
in Z, A, B, C, and D. And that VLAN A traffic (unicast or multicast) has 
to be legally accepted by all
links in A and Z.

There might be some applications that only want to stay within VLAN A. 
But how are RBridges to
know which are supposed to go to Z and which aren't supposed to go to Z?

Radia

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Tue, 3 Apr 2007 16:03:29 -0500

Caitlin,

	I think this is incorrect.  There is little reason to have VLAN 
separation if you are planning to share all information irrespective
of VLAN membership.

	Plus, the example you seem to be referring to does not
necessarily
require the level of information sharing you assume.  There is nothing 
intrinsic in the fact that a IP address is discovered on VLAN B when it 
was earlier discovered on VLAN A that would indicate that the host has
necessarily moved from one to the other.  This could only happen in the
shared IP address space case, but it could happen either because a host 
moves, or because it has presence in both VLANs (as might be the case 
if it was a server, for example).

	Hence I fail to see how it is helpful to share that information.

Thanks!

--
Eric Gray
Principal Engineer
Ericsson  

> [mailto:rbridge-bounces at postel.org] On Behalf Of Caitlin Bestler
> Sent: Tuesday, April 03, 2007 2:30 PM
> To: Radia Perlman

---

From: caitlinb at broadcom.com (Caitlin Bestler)
Date: Tue, 3 Apr 2007 15:06:19 -0700

> [mailto:rbridge-bounces at postel.org] On Behalf Of Silvano Gai
> Sent: Tuesday, April 03, 2007 8:16 AM
> I will go one step further and say that the basic TRILL 
> protocol should not specify an ARP/ND proxy function.
> 
> As the discussion as pointed out, ARP/ND proxy is more a 
> product feature that depends on how the box are packaged. 
> IMHO RBridges will be packaged with additional router, NAT, 
> firewall functionalities. If and where proxy ARP will be 
> implemented will vary.
> 
> After the last call on the base protocol, we can always 
> resume the work and deal with Proxy ARP/ND in a separate document.
> 
> -- Silvano
> 

At the minimum we need to ensure that all RBridges have 
the information that would enable them to efficiently
and reliably act as an ARP/ND proxy.

By the very nature of a proxy it is hard to state that
it MUST do anything. Any proxy will have a limit on the
information that it can cache, and can therefore fail
to act as a proxy on some occassions. So anything that
is a proxy function strikes me as inherently a SHOULD
at most.

That said, all of the arguments on why an RBridge is
valuable also imply that proxying ARP/ND is a vital
part of the RBridge's functionality.

In particular, I believe that the trends towards
server virtualization and physically more compact
servers both will lead to an increase in the logical
size of IP subnets. That makes the role of ARP/ND
proxying more important, and makes it all the more
desirable not to routinely run these queries from
one end of the CRED to the other important.

In a virtualization context I also think it makes
for more sense for each Virtual Machine to be able
to obtain the ARP/ND info for host X when it needs
it without having *each* of them having to do an
end-to-end query or forcing each of them to hear
every ARP response so that each virtual machine
can maintain its own ARP table in parallel.
The RBridge is simply in the ideal position to
perform a highly optimized ARP/ND proxy role.

---

From: jrrivers at nuovasystems.com (J. R. Rivers)
Date: Tue, 3 Apr 2007 15:31:18 -0700

snip... 
> and reliably act as an ARP/ND proxy.

It depends on how you define the requirements of ARP/ND proxy.  I have
seen this general mechanism used in many contexts... only one of which
is covered by an IETF RFC (AFAIK).  Bridges in their basic definition
don't have ARP/ND proxy.  Only bridges that subsume some type of IP
related functionality contain these.

If an RBridge "looks and smells" like a bridge, then there is natural
traffic separation between VLANs, and this allows systems companies to
view RBridges as "better bridges". 

JR

---

From: caitlinb at broadcom.com (Caitlin Bestler)
Date: Tue, 3 Apr 2007 15:38:00 -0700

> Sent: Tuesday, April 03, 2007 3:31 PM
> To: Caitlin Bestler; Silvano Gai; Eric Gray (LO/EUS); Radia 
> Perlman; rbridge at postel.org
> Subject: RE: [rbridge] Two "shared VLAN" alternative proposals
> 
> 
> snip... 
> > 
> > 
> > At the minimum we need to ensure that all RBridges have the 
> > information that would enable them to efficiently and 
> reliably act as 
> > an ARP/ND proxy.
> 
> It depends on how you define the requirements of ARP/ND 
> proxy.  I have seen this general mechanism used in many 
> contexts... only one of which is covered by an IETF RFC 
> (AFAIK).  Bridges in their basic definition don't have ARP/ND 
> proxy.  Only bridges that subsume some type of IP related 
> functionality contain these.
> 
> If an RBridge "looks and smells" like a bridge, then there is 
> natural traffic separation between VLANs, and this allows 
> systems companies to view RBridges as "better bridges". 
> 
> JR
> 
> 

The reason RBridges are "better bridges" is that they
deal with the issues of large subnets far better than
bridges do.

Efficient distribution of ARP/ND information is also
an issue where a "better bridge" is needed to scale
to larger subnets efficiently.

---

From: jrrivers at nuovasystems.com (J. R. Rivers)
Date: Tue, 3 Apr 2007 15:46:42 -0700

I would have thought that RBridges are "better bridges" because they
allow multi-pathing at layer 2 creating significantly more available
bandwidth and with a deployment cost/complexity relatively close to
802.1 bridges.

JR

> Sent: Tuesday, April 03, 2007 3:38 PM
> To: J. R. Rivers; Silvano Gai; Eric Gray (LO/EUS); Radia

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Tue, 3 Apr 2007 16:22:23 -0700

Do we have any quantitatively data about the need of ARP/ND proxy?

With an ARP cache of 30 minutes, typical in hosts today, even with a 100
K hosts in a VLAN we get at most 55 ARP seconds. Since not all the hosts
talk with each other, it is more typically like 5 ARP/sec.

The ARP/ND cache on the RBridge must be significantly shorter than 30
minutes to not increase the amount of obsolete information.

Are we reducing from 5 ARP/sec to 3 ARP/sec for 100 K hosts per VLAN?

Is this the big optimization for which we care to create corner cases
and potential incompatibilities?

-- Silvano

> To: J. R. Rivers; Silvano Gai; Eric Gray (LO/EUS); Radia Perlman;
> rbridge at postel.org

---

From: caitlinb at broadcom.com (Caitlin Bestler)
Date: Tue, 3 Apr 2007 16:36:47 -0700

> Sent: Tuesday, April 03, 2007 4:22 PM
> To: Caitlin Bestler; J. R. Rivers; Eric Gray (LO/EUS); Radia 
> Subject: ARP/ND (was RE: [rbridge] Two "shared VLAN" 
> alternative proposals)
> 
> 
> With an ARP cache of 30 minutes, typical in hosts today, even 
> with a 100 K hosts in a VLAN we get at most 55 ARP seconds. 
> Since not all the hosts talk with each other, it is more 
> typically like 5 ARP/sec.
> 
> The ARP/ND cache on the RBridge must be significantly shorter 
> than 30 minutes to not increase the amount of obsolete information.
> 
> Is this the big optimization for which we care to create 
> corner cases and potential incompatibilities?
> 

The ARP/ND transaction rate is not the critical issue.

The more critical issue is the end-to-end latency of a transaction
that requires doing an ARP/ND first. As the subnet gets larger,
and the number of "L2 Hops" increases, that answer gets worse
unless you implement a distributed ARP/ND proxy service.

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Tue, 3 Apr 2007 16:48:16 -0700

I am surprised,
in ARP the dominant time is the software processing on the two nodes
that goes through a classical interrupt moderation technique and exceed
multiple milliseconds. The Rbridge latency is probably 1 to 10
microseconds, so even if you have 100 hops, it is still negligible.

-- Silvano

> Sent: Tuesday, April 03, 2007 4:37 PM
> To: Silvano Gai; J. R. Rivers; Eric Gray (LO/EUS); Radia Perlman;
> rbridge at postel.org
> Subject: RE: ARP/ND (was RE: [rbridge] Two "shared VLAN" alternative
> proposals)
> 
> 
>

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Wed, 4 Apr 2007 07:11:46 -0500

I agree with all of the reasons Caitlin states below and would add two
more reasons for including ARP/ND as at least a capability:

1) One particular attribute of this work is the use of TTL as a loop
   mitigation technique; an obvious aspect of this approach is its
   total inadequacy in addressing loops in pt-2-mp traffic - hence it
   is imperative that anything that can be done to reduce both the 
   absolute quantity, and the number of paths involved, in pt-2-mp
   traffic is considered to be very important.
2) The TRILL solution - coming as it does from the IETF community -
   needs to be specific to applications of IP (v4 or v6) and therefore
   needs to consider optimization for such applications of primary
   concern.

I would further clarify one of Caitlin's points by saying that it is
pointless to consider defining ARP/ND for later inclusion if we have
not included the basic capabilities in the protocol.  Specifically,
if we have not included the data structures and behaviors necessary
to ensure the distribution of information required to support ARP/ND,
then subsequent work to add these basic capabilities will introduce
potential compatibility issues and limitations.

What all of this seems to mean is that we should define how ARP/ND 
information could be distributed aqnd what information we know has
to be included.  Then we may leave the specification of what to do
with that information to a later date.

--
Eric Gray
Principal Engineer
Ericsson  

> Sent: Tuesday, April 03, 2007 6:06 PM
> To: Silvano Gai; Eric Gray (LO/EUS); Radia Perlman; rbridge at postel.org

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Wed, 4 Apr 2007 07:14:59 -0500

JR,

	I agree that "natural traffic separation between VLANs" is a 
requirement.

	This, however, does not mean that an ARP/ND optimization within
the separate VLANs is in any way unnatural.

--
Eric Gray
Principal Engineer
Ericsson  

> Sent: Tuesday, April 03, 2007 6:31 PM

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Wed, 4 Apr 2007 07:16:05 -0500

And you would be correct.

--
Eric Gray
Principal Engineer
Ericsson  

> Sent: Tuesday, April 03, 2007 6:47 PM

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Wed, 4 Apr 2007 08:06:14 -0500

Silvano,

	It's possible that Caitlin made a mis-statement.  The impact of
a larger propagation radius is in the number of branches that usually
is implied by such an increase.  More branches, more risks of looping 
broadcast and/or multicast traffic, hence more impetus to reduce both.

--
Eric Gray
Principal Engineer
Ericsson  

> Sent: Tuesday, April 03, 2007 7:48 PM
> Subject: RE: ARP/ND (was RE: [rbridge] Two "shared VLAN" 
> alternative proposals)
> Importance: High
> 
> 
> I am surprised,

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Wed, 4 Apr 2007 08:13:37 -0500

Silvano,

	Where IP (v4 or v6) is the exclusive higher layer in a specific
network deployment, it is likely that ARP is one of the most common 
forms of broadcast traffic that may occur.  As we've previously agreed 
that broadcast traffic is of particular concern in TRILL, the potential
to drastically reduce (or possibly eliminate) common broadcast traffic
should be considered a worth-while objective.

--
Eric Gray
Principal Engineer
Ericsson  

> Sent: Tuesday, April 03, 2007 7:22 PM
> with a 100

---

From: jrrivers at nuovasystems.com (J. R. Rivers)
Date: Wed, 4 Apr 2007 06:57:48 -0700

It seems that if ARP is the bulk of your broadcast/multicast, then you
don't really need a very efficient broadcast/multicast mechanism.  Some
networks can be characterized as you've stated; however, there are
others that use L2/IP multicast in a substantial fashion.

JR

> From: Eric Gray (LO/EUS) [mailto:eric.gray at ericsson.com] 
> Sent: Wednesday, April 04, 2007 6:14 AM
> To: Silvano Gai; Caitlin Bestler; J. R. Rivers; Radia

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Wed, 4 Apr 2007 09:33:46 -0500

JR,

	I agree.  However, I believe we (as a working group) have
already
agreed that the IP multicast optimization is essential.  Also, it is my
point (and it is certainly arguable) that perhaps TRILL should not be
targetting the general applications of L2 where other forms of L2 bcast
are likely to be a significant factor - but should instead focus on the
applications of L2 specific to IP only (or IP dominant) networks.

--
Eric Gray
Principal Engineer
Ericsson  

> Sent: Wednesday, April 04, 2007 9:58 AM
> To: Eric Gray (LO/EUS); Silvano Gai; Caitlin Bestler; Radia

---

From: jrrivers at nuovasystems.com (J. R. Rivers)
Date: Wed, 4 Apr 2007 07:36:01 -0700

Why does ARP/ND equate to IP multicast optimization?

JR

> Sent: Wednesday, April 04, 2007 7:34 AM
> To: J. R. Rivers; Silvano Gai; Caitlin Bestler; Radia

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Wed, 4 Apr 2007 09:38:17 -0500

It doesn't.  However, you mentioned IP multicast as another area of
concern... 

Thanks!

--
Eric Gray
Principal Engineer
Ericsson  

> Sent: Wednesday, April 04, 2007 10:36 AM

---

From: jrrivers at nuovasystems.com (J. R. Rivers)
Date: Wed, 4 Apr 2007 08:26:12 -0700

Sorry I must have mis-stated my point.  It seems that if ARP is the bulk
of your multicast traffic, then optimizing it doesn't seem like much
return-on-investment.  If other multicast comprises the bulk, then
optimizing ARP doesn't seem to provide much return-on-investment.

JR

> Sent: Wednesday, April 04, 2007 7:38 AM

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Wed, 4 Apr 2007 11:06:42 -0500

JR,

	Just to be precise, in the working group discussions, we have
been making a very strong distinction between multicast and broadcast.

	I agree that - in an environment where multicast is a large
factor (where large means something like more than 1% of the total
injected traffic), the traffic associated with ARP is way down in 
the noise.  However, if any effective form of multicast optimization 
is implemented, the impact of potentially looping multicast can be 
scoped to affected VLANs (and possibly even to multicast "listeners") 
by techniques other than TTL (such as queue management that provides 
effective isolation of logical interfaces, or hard upper limits on 
total multicast traffic, for instance).

--
Eric Gray
Principal Engineer
Ericsson  

> Sent: Wednesday, April 04, 2007 11:26 AM
> > > > > > that broadcast traffic is of particular concern in 
> TRILL, the 
> > > > > > potential
> > > > > > > K hosts in a VLAN we get at most 55 ARP seconds. 
> Since not 
> > > > > > > all the hosts
> > > > > > > Are we reducing from 5 ARP/sec to 3 ARP/sec for 
> 100 K hosts 
> > > > > > per VLAN?
> > > > > > >

---

From: sgai at nuovasystems.com (Silvano Gai)
Date: Wed, 4 Apr 2007 09:30:21 -0700

Let me try to summarize what I learnt from the replies to my message. 

ARP/ND optimization:
1) it is not justified by a bandwidth issue;
2) it is not justified by a latency issue;
3) may help reduce the problem with the broadcast storm, but the problem
is really IP multicast. Solution like the one proposed by Radia at the
last IETF address both IP multicast and ARP; therefore there is no need
for a special ARP/ND solution;
4) it introduces corner cases.

Why do we want to do ARP/ND optimization in the first version of TRILL?

I restate my proposal: let's drop it by the first version of TRILL and
deal with it later, in a separate document.

-- Silvano

> From: J. R. Rivers
> Sent: Wednesday, April 04, 2007 8:26 AM
> To: Eric Gray (LO/EUS); Silvano Gai; Caitlin Bestler; Radia Perlman;
> rbridge at postel.org
> > > > > > that broadcast traffic is of particular concern in TRILL,
the
> > > > > > > Are we reducing from 5 ARP/sec to 3 ARP/sec for 100 K
hosts

---

From: jrrivers at nuovasystems.com (J. R. Rivers)
Date: Wed, 4 Apr 2007 09:47:57 -0700

I don't disagree... I couldn't make the connection between this and
proxy ARP/ND.

JR

> Sent: Wednesday, April 04, 2007 9:07 AM
> To: J. R. Rivers
> [mailto:jrrivers at nuovasystems.com]

---

From: anoop at brocade.com (Anoop Ghanwani)
Date: Tue, 24 Apr 2007 18:14:18 -0700

> [mailto:rbridge-bounces at postel.org] On Behalf Of J. R. Rivers
> Sent: Tuesday, April 03, 2007 3:47 PM
> I would have thought that RBridges are "better bridges" 
> because they allow multi-pathing at layer 2 creating 
> significantly more available bandwidth and with a deployment 
> cost/complexity relatively close to
> 802.1 bridges.

Is TRILL going to solve the multipathing problem?

From
http://www.ietf.org/internet-drafts/draft-ietf-trill-routing-reqs-02.txt
   However, support for multi-pathing is potentially problematic and is 
   assumed - in this document - to be a non-goal.  Multi-path forwarding
   has the potential to confound the bridge learning process.

Anoop

---

From: Radia.Perlman at sun.com (Radia Perlman)
Date: Wed, 25 Apr 2007 20:57:36 -0700

> Is TRILL going to solve the multipathing problem?
Multipathing comes "for free" because of using layer 3 routing 
technology. Whatever
capabilities IS-IS has for routing to a destination can be used for 
routing to the
egress RBridge.

So the answer is yes.

Radia

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Thu, 26 Apr 2007 13:18:20 -0500

Radia,

	I started to reply to Anoop, but decided to hold of
because this is a complicated discussion and there are more
than enough of those going around.

	The issue is that not all of us mean the same thing
when we say "multi-pathing."  I guess I will fire of my 
response to Anoop after all...

Thanks!

--
Eric Gray
Principal Engineer
Ericsson  

> Sent: Wednesday, April 25, 2007 11:58 PM
> To: Anoop Ghanwani
> Cc: J. R. Rivers; Caitlin Bestler; Silvano Gai; Eric Gray 
> (LO/EUS); rbridge at postel.org

---

From: eric.gray at ericsson.com (Eric Gray (LO/EUS))
Date: Thu, 26 Apr 2007 13:26:12 -0500

Anoop,

	There are (at least) two possible interpretations of what 
is meant by "multi-pathing."

	One is the sort of multipathing that is roughly analogous
to either link-bundling (at L2) or ECMP (usually for L3 routing).

	The other is the sort that has to do with the more general
form of "load-sharing" - i.e. - simply trying to ensure that all
possible links are being used, roughly at some proportional ratio
relative to their capacities.  This form of multi-pathing is not
dependent on multiple-path use for essentially the same traffic
(i.e. - as defined by edge-to-edge unidirectional forwarding).

	I believe that many people (especially those who are not 
completely up to speed on - for example - MSTP) are thinking of
the second form of "multi-pathing" when they refer to this term.
When using MSTP, there are no unused links and load-sharing is
a natural consequence of the assignment of VIDs on the basis of
separate spanning tree instances.

	As you may know, there are problems that must be addressed 
in using the first form of multi-pathing scheme if we need to be
able to maintain forwarding symmetry on an end-point pair basis.  
These difficulties are reduced by using a hierarchical (tunneled) 
overlay with RBridges, but they are not eliminated...

Thanks!
--
Eric Gray
Principal Engineer
Ericsson  

> From: Anoop Ghanwani [mailto:anoop at brocade.com] 
> Sent: Tuesday, April 24, 2007 9:14 PM
> To: J. R. Rivers; Caitlin Bestler; Silvano Gai; Eric Gray 
> (LO/EUS); Radia Perlman; rbridge at postel.org