![]() Making your crawler configurable allows you to adapt it as your repository settings change. By keeping your project as simple as possible, you will make it easier to develop, use, and maintain.Įssential for the crawler to work properly. We recommend that you only implement the capabilities that are relevant to your use case for a custom crawler. The following are typical capabilities of a crawler. For instance, if you want to create a crawler that manipulates JSON, the Newtonsoft Json.NET library should be helpful. In addition, consider the availability of libraries for the operations you want to perform, i.e., the capabilities of your crawler. Many APIs are RESTful, therefore any language that allows you to easily perform HTTP requests should be a good choice. NET SDKs for Microsoft Dynamics CRM, so it might be a good idea to use C# and. To choose the technology with which you will conduct your development project, you must understand how the repository you want to crawl works, and you must have established the capabilities you want to give your crawler.įirst, consider the availability of libraries or SDKs in the programming language of your choice. ![]() Write the other desired capabilities and features. ![]() See the Typical Workflow of a Crawler section below for details. If applicable, begin with the authentication capability. See also the C# SDK and Push API documentation. To try them, you can create a test organization. This will help you determine the scope of your project.Įxplore the APIs or SDKs of the repository you want to index and validate that it can support the desired crawler features.Įxplore Coveo’s APIs and SDKs. ![]() The Easy File Pusher is the most simple example of a crawler.ĭetermine what capabilities you want your crawler to have. For instance, Coveo’s native connectors as well as the Crawling Module all rely on a crawler to discover your data. Project PlanningĪ crawler is a program that automatically discovers content items in a repository. We recommend thoroughly reading this article before starting to write your crawler code, as it will help you make important decisions that can heavily impact the scope of your project. This article is a high-level guide for developers who want to build their own custom connector. To do so, use Coveo’s C# SDK and Push API. If none of Coveo’s connector options fit your needs, you can build your own connector, i.e., write a crawler that retrieves your content and provides it to Coveo for indexing.
0 Comments
Leave a Reply. |