GoAnywhere MFT Overview

This page header displays the GoAnywhere environment name, the system name of the local server and the current Domain.

On each drop-down menu you will find the following components:

• Resources: Resources are the names and connection properties of the servers.

• Workflows: Triggers and Projects which automate any file transfers and business processes.

• Services: GoAnywhere Services are used for inbound connections from your trading partners, customers, employees, and remote sites.

• Users: Roles, Login Methods, Admin Users and Admin Groups to control access.

• Logs and Reports: Errors and monitoring events such as file transfers and server activity.

• Encryption: GoAnywhere implements several popular encryption technologies for securing data.

• System: Provides access to change many of the settings for how GoAnywhere operates.

We are going to explain a little more the main GoAnywhere MFT components which will help you to understand how this technology works.

3.1. Resources

Resources are the names and connection properties of the servers, and other data sources, that GoAnywhere can interact with. Here you have a picture of the main GoAnywhere resource web console:

Figure 6. Resources web console.

You can create your own resources there, test connections, import/export resources using XML templates and manage resource permissions.

We are going to list the most important resources from our point of view:

• Amazon S3 Buckets: The names and connection properties of Amazon S3 Bucket locations to which GoAnywhere can connect for accessing files.

• Network Shares: The names and connection properties of shared file locations on a network to which GoAnywhere can connect for accessing files. We quite often use it to connect to Samba servers.

• PGP Key Rings: The names of the PGP Key Rings that can be used in GoAnywhere to encrypt and decrypt files using the OpenPGP standard.

• SSH Servers (SFTP/SCP/SSH): The names and connection properties of the SSH Servers which GoAnywhere can connect to. These resources can be used for SFTP transfers (SSH File Transfer Protocol), SCP file transfers (Secure Copy) and running remote SSH commands.

Finally, the aim of developing these resources is to use them later inside the GoAnywhere MFT logics and triggers, so not forget setting up them correctly and test them!

3.2. Workflows

You can develop easily the desired logic inside GoAnywhere MFT projects to describe the work to perform. These projects can be executed immediately or scheduled to run at future dates and times, even run when a certain event occurs.

All these elements related to the GoAnywhere MFT logic are stored inside Workflows. We are going to explain a little more the main items which we usually use inside our integrations.

3.2.1. Projects

Projects can be created in GoAnywhere to automate any file transfers and business processes (workflows) for your organization.

Several methods are available for executing projects within GoAnywhere. You can execute projects on-demand from within GoAnywhere’s browser-based administrator, also be executed by the integrated Scheduler and File Monitor processes. External applications can also launch projects from other systems (for example, IBM i, Windows, Unix, Linux, Solaris, etc.) by using GoAnywhere’s commands and APIs.

The main actions you can do on a MFT Project are: Create a new project, Search projects, Import a project developed using a xml template, Export an existing project, and Edit an existing project.

Projects can be stored within user-defined folders in GoAnywhere. Folders are very useful for organizing projects, which will become important as you create more and more projects. Indeed, if GoAnywhere is configured to use multiple Domains, each Domain name will appear at the root level of the Project Explorer as a folder.

Let’s continue talking about the project design on GoAnywhere MFT. Projects are made up of Modules, Tasks and Elements:

• A Module is a logical grouping of one or more Tasks. For instance, a module may be defined with three Tasks to be executed in sequential order. The first Task in the Module may read (parse) data from an XML document. The second Task may insert that data into a database file. The third Task could then call a program to process that data. Multiple modules can be defined in a project. The Call Module task can execute another Module in a project to perform a sub-routine of tasks based on specific criteria.

• GoAnywhere MFT also include Control Structures tasks inside its projects such us If Condition, Else and Loops.

• Another quite important tasks over a file are Copy, Move, Delete, CreateFileList. Note that the tasks Copy and Move allow us to rename the input file processed inside the destination on an easy way, which is quite useful for some scenarios.

• A task is made up of one or more Elements, which describes the types of work to perform within the Task. For example, the FTP task has an Element to put a file on a FTP server, an Element to get a file from a FTP server, an Element to rename a file on a FTP server, and so on.

Finally, here you have an example of Modules, Tasks and Elements:

Figure 7. Modules, Tasks and Elements.

3.2.2. Schedulers

GoAnywhere allows you to schedule project execution (Jobs) at future dates and times. A Scheduled Job can be created or edited using the Scheduled Job page. You are able to add a new Schedule and to edit an existing one.

Inside a Schedule, you can set up the frequency, target project, run priority or even use a recorded Holiday Calendar to determine what should happen to the job when the execution falls on a holiday.

3.2.3. Monitors

The File Monitors function in GoAnywhere provides the ability to repeatedly scan for new, changed or deleted files in a folder. When the scan condition is met, the Monitor can call a pre-defined project to process the file(s). Therefore, this component will help you to develop integrations following the event driven approach.

Figure 8. File Monitor function in GoAnywhere.

You can Create, View, Edit, Delete and Deactivate/Activate monitors.

To create or modify a Monitor, you are able to modify fields such as the source resource and folder to check out periodically, the target MFT Project to trigger and its input parameters, the file Event Type (File Created or Modified, File Created, File Modified, File Deleted, and File Exists), and the file name Pattern Type (Wildcard or Regular Expression). Of course, you can set up the Monitor schedule modifying its start time, end time and frequency.

4. Business Scenario Example

You have already seen a lot of information about the features of how the GoAnywhere MFT platform. Let’s see an example which will help you achieve a better understanding of the real value of this technology. The example scenario is as follows:

I need to move files from a source sftp server to a target sftp server.
The source directory will be polled every 1 minute to check for files with a GO extension.
If found, the associated file (without GO extension) and the GO file will moved to a target server directory.
All files moved also need to be moved to a ‘processed’ subfolder to enable replay.

The diagram below shows the overall solution design:

Figure 9. Solution design.

You can see from the diagram the next GoAnywhere MFT components:

1. A SFTP MFT resource connected to the source SFTP server where the files will be pulled.

2. A SFTP MFT resource connected to the target SFTP server where the input csv files must be moved.

3. A MFT Monitor which is using the source SFTP resource to check out input files with .GO prefix.

4. The first MFT Project which is triggered by the monitor when the file pattern is met.

5. A second MFT Project (surrounded by the red square) which is triggered by each detected file and will move the associated csv to the target SFTP resource.

Implementing this solution, you will have an automated job which will suit the business requirements with the best performance, especially if we deal with large files.

5. How can I develop GoAnywhere MFT solutions?

The most straight forward way to develop MFT solutions is through the GoAnywhere MFT Console. However, if you are already familiar with this technology, we would suggest import your MFT xml templates through the GoAnywhere Command Web Services.

5.1. Using the GoAnywhere MFT Console

This is the best option if you are starting working with GoAnywhere MFT. As previously discussed you are able to create all the desired MFT items for your integration meets the business requirements.

Remember that a project can be created from scratch by choosing the specific Tasks needed, or you can create projects by using one of the included GoAnywhere templates. Using a template is generally the fastest approach for creating a project, since each template contains a pre-defined group of Tasks for performing commonly used business processes.

Figure 10. Database to Excel to FTP.

5.2. GoAnywhere Command Web Services

If you are use to working with MFT and you want speed up your development and deployment, you may want to import your MFT items templates through the GoAnywhere Command Web Services. This allows users to perform SOAP and REST requests to a local or remote GoAnywhere system. With GoAnywhere Command Web Services, it is possible to execute GoAnywhere functions from custom programs, and import your custom MFT components. The following example that shows how to import a project from an XML file:

Figure 11. Import a project from an XML file. 

Indeed, we have automated the testing and deployments of our GoAnywhere MFT items using continuous methodologies (such as gitlab-ci and Jenkins) and gradle tasks which call to the GoAnywhere Command Web Services.

Also, remember almost all the GoAnywhere components are defined by xml template files. We usually use Visual Studio Code IDE to work with our MFT integrations because it provides a lot of plugins for xml templates files, but other IDEs are just as valid. Here you have some template examples where you are able to see all the main properties for each GoAnywhere MFT item:

• This is an XML file which defines a MFT Project:

<project name="Example Using Variables" mainModule="Main" version="2.0" logLevel="verbose">
<variable name="MyFile" value="" />

    <module name="Main">

        <sftp label="Connect to Server" resourceId="Test SFTP" version="1.0">
            <put label="Put file on Target Server" sourceFile="${MyFile}" destinationDir="/Demos/inbound" />
        </sftp>

    </module>

    <description>Basic Project with Parameter</description>
    
</project>

• This is an XML file which defines a MFT SFTP resource:

<?xml version="1.0" encoding="UTF-8" ?>
<resource class="com.linoma.ga.projects.resources.SSHResource">
    <primaryContactName />
    <primaryContactPhone />
    <primaryContactEmail />
    <secondaryContactName />
    <secondaryContactPhone />
    <secondaryContactEmail />
    <resourceName>{{ resourceName }}</resourceName>
    <domainName>{{ domainName }}</domainName>
    <resourceDescription>{{ resourceDescription }}</resourceDescription>
    <host>{{ host }}</host>
    <port>{{ port }}</port>
    <user>{{ userName }}</user>
    <password>{{ password }}</password>
    <passwordIsEncrypted />
    <passwordIsRecoverable />
    <keyLocation>KeyVault</keyLocation>
    <keyVault>{{ domainName }}</keyVault>
    <privateKeyAlias>{{ privateKeyAlias }}</privateKeyAlias>
    <keyPassword />
    <keyPasswordIsEncrypted />
    <keyPasswordIsRecoverable />
    <privateKeyFile/>
    <privateKeyPassword/>
    <privateKeyPasswordIsEncrypted />
    <privateKeyPasswordIsRecoverable />
    <rsaSignatureAlgorithm>ssh-rsa</rsaSignatureAlgorithm>
    <hostKey />
    <timeout>{{ timeout }}</timeout>
    <connectionRetryAttempts>{{ connectionRetryAttempts }}</connectionRetryAttempts>
    <connectionRetryInterval>{{ connectionRetryInterval }}</connectionRetryInterval>
    <initialRemoteDirectory>{{ initialRemoteDirectory }}</initialRemoteDirectory>
    <throttleBandwidth />
    <proxyType />
    <proxyHost />
    <alternateProxyHost />
    <proxyPort />
    <proxyUser />
    <proxyPassword />
    <proxyPasswordIsEncrypted />
    <proxyPasswordIsRecoverable />
    <resolveAtProxy />
    <authenticationAlgorithms />
    <cipherAlgorithms />
    <macAlgorithms />
    <keyExchangeAlgorithms />
    <compressionAlgorithms />
</resource>

• This is an XML file which defines a MFT Monitor for a SFTP server:

<?xml version="1.0" encoding="UTF-8" ?>
<monitor class="com.linoma.dpa.commandcenter.codec.MonitorVOCodec">
  <status>{{ status }}</status>
  <actionType>0</actionType>
  <triggerDefinition>{{ triggerDefinition }}</triggerDefinition>
  <description>{{ description }}</description>
  <domain>{{ domainName }}</domain>
  <name>{{ monitorName }}-{{ domainName }}</name>
  <monitorType>3</monitorType>
  <callProject jobName="{{ jobName }}" onFailureAttachJobLog="true" onSuccessAttachJobLog="false" password="{{ password }}" passwordIsEncrypted="false" runPriority="5" user="{{ runner }}" project="{{ projectLocation }}">
    <variable originalValue="" value="${monitor.fileList}" name="monitorFileList" description="files set by monitor" />
  </callProject>
  <ftpMonitorDefinition>
    <resourceName>{{ sftpOriginResourceName }}</resourceName>
    <resourceType>ssh</resourceType>
    <folder>{{ sftpOriginFolderName }}</folder>
    <recursive>false</recursive>
    <stopChecking>{{ stopChecking }}</stopChecking>
    <dualListDelay>10</dualListDelay>
    <events>
      <event type="{{ eventType }}" />
    </events>
    <filters>
      <regexFilter>
        <include pattern="{{ filePattern }}" caseSensitive="false" />
      </regexFilter>
    </filters>
  </ftpMonitorDefinition>
</monitor>

6. Conclusion

In this article, we have shown you an overview of the GoAnywhere MFT platform. You know now the main features of this technology, it’s architecture/structure, how to install on your environment and how to start working with it.

In addition, you have seen here an example scenario and some suggestions of how to develop your GoAnywhere MFT integrations.

In our experience, GoAnywhere is a good solution to automate the procedure of moving files from a source to a target. Indeed, we have detected in our daily work that GoAnywhere MFT resolves that requirement with a better performance than other technologies (custom Java applications for instance) especially if we are talking about working with large files. However, we generally use other tools where the requirement is to transform or read data from files.

Therefore, we encourage you to start using GoAnywhere MFT if you have to implement a file transfer solution.

For detailed insights on how GoAnywhere MFT can fit within your organisation, please don’t hesitate to get in touch with Chakray. Our team is ready to provide expert support on how you can make this powerful solution worthwhile to you. Your journey to efficient, secure, and hassle-free file transfers begins with us.

Need help defining and implementing your integration strategy for your company?

Talk to our experts!

contact us about integration