ofproto: Add "fast path".
The key to getting good performance on the netperf CRR test seems to be to
handle the first packet of each new flow as quickly as possible. Until
now, we've only had one opportunity to do that on each trip through the
main poll loop. One way to improve would be to make that poll loop
circulate more quickly. My experiments show, however, that even just
commenting out the slower parts of the poll loop yield minimal improvement.
This commit takes another approach. Instead of making the poll loop
overall faster, it invokes the performance-critical parts of it more than
once during each poll loop.
My measurements show that this commit improves netperf CRR performance by
24% versus the previous commit, for an overall improvement of 87% versus
the baseline just before the commit that removed the poll_fd_woke(). With
this commit, ovs-benchmark performance has also improved by 13% overall
since that baseline.