Network Information

Some of the information in this section is repeated elsewhere in the resource guide, but we have tried to pull all the network related information into one spot to make it easier for the person in charge of the network to understand Fusion’s requirements.

Internet and Main Router

It is necessary for the feedlot to have a constant internet connection. Fusion uses the internet for many things including communicating with services such as CCIA and NLIS, checking for new versions and downloading install files, sending server health data and crash information to us, sending Lot Info Export files each night when requested, and many other things.

Generally speaking, we do not recommend that Fusion connect over the internet (although see below for a discussion regarding connecting to satellite yards over the internet). This means that a local network will need to be built out, whether wired or wireless, to cover the barns and trucks.

We recommend that all Fusion related computers be on the same subnet, including computers at other locations. It is possible for Fusion to work when this is not the case, but manual entering of IP addresses during connection will often be necessary. If everything is on the same subnet Fusion can usually automatically detect the server and connect automatically.

Office Portion

Within the office we recommend a wired gigabit connection between the server and all computers.

Chuteside Portion

While a wired gigabit connection from the office to each chuteside computer would be ideal, this isn’t usually feasible. Good point-to-point wireless radios in the 2.4GHz and 5.8GHz ranges with 100-1000 Mbps bandwidths can be installed and are recommended if wired is not feasible.

Fusion Client requires a stable, constant connection to work. Low latency and high bandwidth characteristics are critical, so care should be taken in choosing the equipment to spread your network to the chuteside areas.

Feed Truck Portion

Radios similar to those used at chuteside are excellent for truck use, but not as necessary. The truck only needs to be connected to the network during the syncing process. Ideally this will occur after every load and only takes a few seconds. In this case it is sufficient to establish a wireless connection point near the mill, for example, where the truck can sync at known connection points.

If a system such as this is not feasible, it should be possible to use off-the-shelf wireless products, perhaps driving close to the office for a connection and syncing less often.

We recommend that a long length of ethernet cable be prepared so that if the wireless network is not working some day, it would be possible to connect the truck computer to the office switch through a door or window so that syncing could be accomplished at least morning and night until the issue is fixed.

Static IP Addresses

The computer that Fusion Server runs on must have a local static IP address. All other computers may use DHCP, if desired.

Firewall Settings

The server needs to have TCP ports 19812-19814 and 19820 as well as UDP ports 19813 and 19820 opened. If Symmetry will be used, TCP port 19850 should also be opened. It is critical that the ports themselves are opened rather than just allowing the Fusion Server application to receive incoming requests. If you use the latter mechanism, it will appear to work but will fail in the future with new versions of Fusion.

Purchasing Network Equipment

It has been our experience that network instability is one of the greatest causes of frustration for Fusion users (over 75% of errors are caused by network instability). It seems that many of the cheaper devices (routers, switches, etc.) and cables often don’t last very long, especially in a demanding feedlot environment.

We recommend spending a little more here to get higher quality equipment and consider having backup equipment on hand and pre-configured.

If you can add the network equipment to your surge protector and battery backup devices, this is best.

Networks Requirements

Important: You must test your network using the tools in Fusion Installer to know if your network aligns with the following recommendations.

We recommend 50 Mbps or higher for office computers. Greater than 100 Mbps is preferred, where possible. For chuteside computers, we recommend 10 Mbps or higher. Trucks can be as low as 1-2 Mbps for syncing, but upgrades work best with a minimum of 10 Mbps. Anything less than these recommendations will likely result in sluggish performance at best and many disconnects at worst.

Latency should be minimal and affects Fusion Client performance much more than bandwidth. Ideally office computers will be <= 1 ms. Chuteside computers can be higher, but ideally less than 5 ms (anything more than 10 ms will make some chuteside functions very slow). Truck computers are not really affected by latency, so we don't have a specific requirement for them. Packet loss should basically be 0% for all computers. Anything outside these parameters should be corrected.

The chart below shows the effect that throughput and latency in the network can have on Fusion and is included to give you a better idea of the importance of network requirements. Please note that some of the functions in the chart have been further optimized since this data was collected, so don't take this as an indication of how fast a specific feature should be. The point of the chart is to show what effect latency can have.

If you have computers on your network that do not meet the minimum requirements, please work to address the issue. This may include changing settings, moving equipment around, or even upgrading to faster equipment.

For your information, more details on some of the tests in the chart above are provide here.

  • Test B: Moved a ~70 MB file from Fusion Server to Fusion Truck.
  • Test D: Shows the combined time to gather and save information for 255 animals in a job.
  • Test E: Opened an Animal History window and loaded all sections.
  • Test F: Clicked the Refresh button in the Daily Feed Detail window.
  • Test H: Scrolled through 694 lots in the Lot Center window with all default columns in the view.
  • Test I: Scrolled through 694 lots in the Lot Center window with a custom view containing 12 columns.
  • Test J: Generated a complex yard report from the Pen List window.

Satellite Yards

Some feedlots have satellite yards that cannot easily be connected to the main yard where the server is located without going through the internet, due to distance or terrain. While we can't guarantee Fusion will work out of the box for these satellite yards, it is worth discussing the circumstances that could make it feasible. Let's start by reviewing the relevant principles.

Latency

The perceived responsiveness for most of Fusion's features is affected by latency far more than bandwidth. Latency is created by three factors:

  1. Distance. Physics dictates the minimum, theoretical time it takes for a packet to travel a certain distance. The further away the satellite yard is from the server, the longer it will take the packet to travel there. The distance to consider is the actual, physical path the packet must take which will not be as the crow flies. Packets often travel to distant cities on their way to their destination.
  2. Medium. This factor will have a far smaller effect in this context than the other two, but is included for completeness. A packet will travel at slightly different speeds through different mediums. For example, latency will generally be less for the same distance when the signal propagates through fiber as compared to copper or air.
  3. Routing. It is easy to forget how much time routing or switching adds to latency. When a packet arrives at a network device, the device needs to receive it, inspect the header to see what the destination is, look up information about this destination in its tables, possibly rewrite part of the packet, and then send the packet to the correct place on the physical layer. The time it takes to do all this is affected by the device's CPU, RAM, internal algorithms, size of look up table, current congestion and load, prioritization schemes, etc. Each device a packet must pass through increases the latency, sometimes dramatically.

Routing

For a satellite yard to communicate with Fusion Server at the main office, there has to be a way for network packets to know how to get to the server. There are basically two ways to achieve this:

  1. Public Static IP. This solution usually varies between somewhat difficult to impossible in rural areas, depending on whether ISPs are willing (or even able) to give out IPv4 static IP addresses. The main yard must be given a publicly accessible IP address with the router set to forward all TCP traffic for ports 19812-19814 and 19820 to the internal IP address of the machine Fusion Server is running on. At the satellite yard, Fusion must be set to connect to Fusion Server using the public IP address. While this will logically work, it additionally has the same downside as running Fusion on different subnets—the user will need to manually enter the IP address from time to time.
  2. Virtual Private Network. VPNs (or similar solutions such as Tailscale) allow computers to be configured so that, from a network point of view, they appear to be on the same local network as another computer even though they are connecting across the internet. Although care needs to be taken to configure VPNs correctly for a given situation, they tend to overcome the downsides of using a public IP address while having the same benefits.

Requests per Function

The last principle affecting the feasibility of satellite yards revolves around the number of network requests that are required for each function in Fusion. Each time Fusion needs a piece of information from the server a packet must be sent across the network to the server. Then the reply must come back to Fusion. If the latency is 100 ms, it will take 200 ms for the response to get back. If 10 requests are made as Fusion gathers the information it needs, it will take a total of 2 seconds before it has all the information it has.

With these principles in mind, let's turn the discussion to what can be done to make a satellite yard connection as responsive as possible.

Network Improvements

When building and configuring the network path between the satellite yard and the main office, reduce latency as much as possible. Consider these ideas:

  • Reduce the physical distance of the logical network path as much as possible. Are you using the same ISP on both ends? They may be able to configure their equipment so packets take the shortest path rather than coming all the way back to their main equipment or up to their provider before coming back. Are you using a VPN? See if you can configure it to use the closest VPN server or consider hosting your own so that you can place it physically as close as possible to the two yards. Be hyper aware of the actual distance packets must travel and find ways to reduce this.
  • Reduce the number of hops in the logical network path as much as possible. Each device a packet must pass through, whether under your control or the ISPs control, adds substantially to the overall latency. Remove unnecessary switches and consider hand tuning the path Fusion packets take. Work with your ISP to do the same where possible.
  • Reduce the time it takes for network devices to route a packet. For the devices you have control over, ensure the hardware specs are high enough to ensure packets don't have to wait to be routed and that the routing process is as fast as possible. Handcraft the routing rules if it will speed things up. Give priority to packets on Fusion ports. For devices you don't have control over, work with the ISPs as much as possible to prioritize Fusion packets.

Fusion Usage Patterns

When the network has been fine-tuned as much as possible, there are three things that should be considered for Fusion to run as optimally as possible at a satellite yard.

  • As seen in the chart above, the responsiveness of list windows is greatly affected by the number of columns being shown. Some list windows, such as the Lot Center and Animals list windows, show many columns by default. You'll get much better performance if you customize these lists to only show a few columns. Set these reduced column lists to be the default when a window opens.
  • Don't expect to use office functions at satellite yards as most of them will not be optimized for a high-latency network.
  • When there is enough of a need (and if possible), we try to optimize chuteside features for high-latency networks by bundling multiple requests in one large request. Please report chuteside-related functions that are running too slowly and we will optimize them in a future version of Fusion, if possible.

To summarize, Fusion wasn't initially designed to work well over the internet, but as the internet in rural areas improves and as we optimize relevant features for this situation, it is becoming more feasible for certain aspects of Fusion to work in satellite yards. Fusion Truck works really well across the internet. Fusion Client can work reasonably well across the internet in a chuteside context for most chuteside functions as long as the network can be set up as discussed above. We do not recommend a Fusion Client dedicated to running office tasks be used across the internet. Some features will work fine, but many will be frustrating to use.

If you choose to use Fusion over the internet at a satellite yard, you'll need to work closely with your network provider to achieve an acceptable level of responsiveness. You'll likely need to be patient during the fine-tuning process. You will also need to work closely with the Fusion support team if you identify functions that haven't been optimized and be patient as we work to improve them in future versions of Fusion.

Network Troubleshooting

General network troubleshooting is beyond the scope of this page, but there are a few points that are particularly relevant to Fusion Client that may be helpful.

Fusion Client doesn't recover from a network drop as well as, say, a web browser. Browsers are simply loading resources. The order the resources are returned doesn't really matter. HTTP servers are usually stateless. If something happens to a connection, the browser can just try again. None of this is true for Fusion Client which works on a very different paradigm. Unfortunately, this means that if Fusion Client errors out due to a network issue, the safest thing to do is to force quit it and relaunch it. Thus it is important to create networks that are as stable as possible.

About 75% of Fusion errors are due to network issues. Interestingly, we have yards that almost never experience a network-related error and we have yards that experience hundreds of them each month, leading us to understand that the network configuration, setup, and hardware has the greatest impact on these errors.

Some things to consider when looking for the reason for network drops:

  • The more network devices there are between the client and the server, the more chance there is of an issue. Reduce where possible.
  • With wireless networks, computers on the fringe of receptivity have an exponentially higher chance of dropping packets. Encourage computer and device placement well within the signal envelope.
  • Some routing/switching devices aggressively kill long-running TCP connections, sometimes in as short a time as 60 seconds or five minutes. This may make sense for some network traffic, such as browsers, but is not good for Fusion traffic. Where possible, change this setting on all devices so they never kill TCP connections.
  • When troubleshooting, make sure you understand what is going on at each device. Did a packet get dropped? Why? Did the device have to restart a service? Did it have to completely restart itself? If so, why? Was it purging memory and that took too long? Make sure you are gathering enough logs at each device to see what is happening at the time of the drop. Don't forget to consider the network stack on the client and server computers when doing this analysis.