Analysis of Intrusion Detection Systems (IDS)

Analysis of aggression detective work establishments (IDS)IntroductionIntrusion detection schemas (IDS) were developed in 1990s, when the profit hackers and worms appe atomic number 18d, initi each(prenominal)y for the identification and re looking of such(prenominal) struggles. The infringement detection systems didnt keep up the ability to stop such attacks rather than detecting and reporting to the ne dickensrk personnel.The Intrusion Prevention Systems got both characteristics i.e. scourge detection and prevention. The detection touch psychoanalyzes the events for all possible threats succession the intrusion prevention stops the spy possible threats and reports the network administrator.Purpose ScopeThe master(prenominal) purpose of the project is to evaluate the trade protection capabilities of incompatible tokens of IDPS technologies in maintaining the network security. It provides detail nurture about the different classes comp binglents of IDPS technolo gies, for example, detection methods, security capabilities, prevention capabilities internals of IDPS. It is primarily foc employmentd on different detection techniques responses by these technologies.1.2 AudienceThe instruction put up be useful for computer network administrators, network security personnel, who have little knowledge about these IDPS technologies.1.3 Project StructureThe project is organized into the following major structure piece 2 provides a general door of IDPS.Section 3 provides detail development about of IDPS technologies, components architecture, detection methodologies, security capabilities prevention capabilities.Section 4 provides the internals of IDPS incident response.Section 2 Introduction of IDPSThis Chapter Explains the Intrusion detection Prevention Process, Uses, Functions and Different Types of IDPSThe modern computer networks provide fast, reliable and critical information not only to small group of people nevertheless also to ever expanding group of drug users. This need led the development of redundant links, note book computers, wireless networks and many others. On one side, the development of these impertinent technologies increased the importance and value of these access services and on other side they provide more paths to attacks.During the past, In the presence of firewalls and anti-virus package, organizations suffered huge losses in minutes to their businesses in terms of their confidentiality and availability to the legitimate clients. These modern threats highlighted the need for more advance protection systems. Intrusion detection prevention systems atomic number 18 intentional to protect the systems and networks from any unauthorized access and damage.An intrusion is an active sequence of related events that deliberately try to cause harm, such as rendering system unusable, accessing unauthorized information or manipulating such information. In computer terminology, Intrusion detection i s the process of remindering the events in a computer network or a host re origin and analyzing them for signs of possible incidents, deliberately or incidentally. The primary functions of IDPS argon the identification of incident, logging information about them, taenia them preventing them from ca employ any damage. The security capabilities of IDPS can be divided into three main categories spying Identification of vixenish attacks on network host systemsPrevention stopping of attack from executingReaction Immunization of the system from future attacks.On the basis of location and type of events they monitor, there are two types IDPS technologies, host- base network establish. The network-based IDPS monitors traffic for picky network segment and analyze the network application protocol occupation for suspicious events. It is commonly deployed at the borders amid networks. While on the other hand, host-based IDPS monitors the activity of a single host and events occurring within that host for suspicious activity.There are two complementary approaches in detecting intrusions, knowledge-based approach and doings based approach. In knowledge-based approach an IDPS looks for circumstantial traffic casts called Signatures, which indicates the malicious or suspicious content while in the behavior-based approach an intrusion can be detected by observing a deviation from normal or unexpected behavior of the user or the system.What is an IDS?The Intrusion Detection Systems (IDS) can be define as tools, methods resources to identify, assess report unauthorized or unapproved network activity.It is the ability to detect attacks against a network or host and sending logs to concern encourage providing the information about malicious attacks on the network and host resources. IDSs fall into two main categoriesHost-Based Intrusion Detection System (HIDS) A HIDS system assume some software that resides on the system and can scan all host resources for acti vity. It will log any activities it discovers to a secure informationbase and check to go steady whether the events barrack any malicious event record listed in the knowledge base.Network-Based Intrusion Detection Systems (NIDS) A NIDS system is usually inline on the network and it analyzes network bundles looking for attacks. A NIDS receives all softwares on a particular network segment via one of several methods, such as taps or port mirroring. It carefully reconstructs the streams of traffic to analyze them for patterns of malicious behavior.The basic process for IDS is that it passively collects entropy and preprocesses and classifies them. Statistical psychoanalysis can be done to determine whether the information falls outside normal activity, and if so, it is then matched against a knowledge base. If a match is found, an alert is sent. Figure 1-1 outlines this activity.Response jitneyGUIHost SystemPre-processingStatistical AnalysisAlert ManagerKnowledgeBaseLong-Term St orageSignature twinnedFig 1.1 Standard IDS SystemWhat is an IPS?IPS technology has all capabilities of an intrusion detection system and can also attempt to stop possible incidents. IPS technologies can be differentiated from the IDS by one characteristic, the prevention capability. erstwhile a threat is detected, it prevents the threat from succeeding. IPS can be a host-based (HIPS), which work best at protecting applications, or a network-based IPS (NIPS) which sits inline, stops and prevents the attack.A typical IPS performs the following actions upon the detection of an attackIPS terminates the network conjunctive or user session.It blocks access to target .i.e. IP address, user account or sever.It reconfigures the devices i.e. firewall, switch or router.It replace the malicious portion of an attack to make it benignAn IPS typically consists of four main componentsTraffic Normalizer Interpret the network traffic and do packet analysis and packet reassembly traffic is fed int o the detection engine service scanner.Service Scanner Builds a reference table that classifies the information helps the traffic shaper manage the flow of the information.Detection Engine Detection engine does pattern matching against the reference table.Figure 1.2 outlines this processResponseManagerGUITraffic NormalizerSystem ScannerDetection EngineAlert ManagerReference TableLong-Term StorageSignatureMatchingFIG 1-2 Standard IPSUses of IDPS TechnologiesThe identification of possible incidents is the main focus of an IDPS, for example, if an intruder has successfully compromised a system by exploiting the vulnerability in the system, the IDPS could report this to the security personnel. Logging of information is another important function of IDPS. This information is vital for security people for upgrade investigation of attack. IDPS has also the ability to identify the rapine of security policy of an organization which could be intentionally or unintentionally, for example, an unauthorized access to a host or application.Identification of reconnaissance activity is one of the major capabilities of IDPS, which is the indication of an imminent attack, for example, scanning of hosts and ports for launching further attacks. In this case, an IDPS can either block the reconnaissance activity or it can motley the configurations of other network devicesFunctions of IDPS TechnologiesThe main difference between different types of IDPS technologies is the type of events they can recognize. Following are some main functionsRecording of information regarding spy events, this information could be stored locally or could be sent to the logging horde.Sending of alerts is one of the vital functions of IDPS. Alerts are sent through different methods i.e. email, SNMP traps, syslog messages and so onIn case of detection of a new threat, some IDPS do have the ability to change their security visibleness, for example, when a new threat is detected, it big businessman be able to collect more detail information about the threat.IDPS not only performs detection nevertheless it also performs prevention by stopping the threat to succeed. Following are some prevention capabilitiesIt can stop the attack by terminating either network connection or user session, by blocking access to a target host.It could change the configuration of other network devices (firewalls, routers switches) to block the attack or disrupt it.Some IDPS could change the table of contents of a malicious IP packet, for example, it can replace the header of an IP packet with a new one.Types of IDPS TechnologiesIDPS technologies can be divided into following two major categoriesNetwork-Based IDPSHost-Based IDPSNetwork-Based IDPSNetwork-based IDPS monitors network traffic for a particular network segment. They analyze the network and application protocol activity to identify any suspicious activity.A network based IDPS is usually sits inline on the network and it analyzes network p ackets looking for attacks. It receives all packets on a particular network segment, including switched networks. It carefully reconstructs the streams of traffic to analyze them for patterns of malicious behavior. They are equipped with facilities to log their activities and report or alarm on questionable events. Main strengths of network-based IDPS arePacket Analysis Network-based IDPSs perform packet analysis. They examine headers of all IP packets for malicious contents. This helps in detection of the common denial of service (DOS) attack. For example, LAND attack, in which both the source destination addresses and source destination ports are same as of the target machine. This cause the target machine to open connection with itself, causing the target machine either performs easy or crash. It can also investigate the payload of an IP packet for specific commands.Real Time Detection Response Network-based IDPS detects attacks in real m as they are occurring in the real ti me and provides faster response. For example, if a hacker initiated a TCP based DoS attack, IDPS can drop the connection by sending a TCP reset.Malicious Content Detection Network-based IDPS remove replaces suspicious portion of the attack. For example, if an email has infected attachment, an IDPS removes the infected file and permits the unclouded email.Evidence for Prosecution Network-based IDPS monitors real time traffic and if an attack is detected and captured the hacker cannot remove the evidence. Because the captured attack has selective information in it but also the information about his or her identification which helps in the prosecution.Host-Based IDPSA Host-Based system monitors the characteristics of a single host and the events occurring within that host for suspicious activity. It require some software that resides on the system and monitors the network traffic, syslog, processes, file access modification and configuration or system changes. It logs any activities it discovers to a secure database and check to see whether the events match any malicious event record listed in the knowledge base. Some of the major strengths of Host-Based IDPS are as underVerification of Attack Host-based IDPS uses logs which contains events that have genuinely occurred. It has the advantage of knowing if the attack is successful or not. This type of detection is more accurate and generates fewer sham alarms.Monitoring of Important Components Host-Based IDPS monitors key components for example, executables files, specific DDLs and NT registry. All of these can cause damage to the host or network.System Specific Activity Host-based IDPS monitors user and file access activity. It monitors the logoff or login procedure and monitors it on the basis of current policy. It also monitors the file access for example, opening of a non shared file.Switched Encrypted Environments Host-Based IDPSs provide greater visibility into purely switched environment by residing on as many critical hosts as needed. Encryption is a challenging problem for network-based IDPS but not a major problem for host-based IDPS. If the host in question has log-based analysis the encryption will have no impact on what goes in to the log files.Near Real Time Detection A host-based IDPS relies on the log analysis which is not a true(p) real time analysis. But it can detect respond as soon as the log is written to and compared to the active attack signatures.Real Time Detection Response Stack-based IDPS monitors the packets as they transverse the TCP/IP stack. It examines inbound outbound packets and examines in real time if an attack is cosmos executed. If it detects an attack in real the time then it can responds to that attack in the real time.Section 2 IDPS Analysis SchemesIDPSs Perform Analysis This Chapter is about the Analysis Process- What Analysis does and Different Phases of Analysis.2.2 AnalysisIn the context of intrusion detection prevention, analysis is th e organization of the essential parts of data and their relationships to identify any anomalous activity of interest. Real time analysis is analysis done on the fly as the data travels the path to the network or host. The fundamental goal of intrusion-detection prevention analysis is to improve an information systems security.This goal can be further broken down pat(p)Create records of relevant activity for follow-up.Determine flaws in the network by detecting specific activities.Record unauthorized activity for use in forensics or criminal prosecution of intrusion attacks.Act as a deterrent to malicious activity.Increase accountability by linking activities of one respective(prenominal) across system.2.3 Anatomy of Intrusion AnalysisThere are many possible analysis schemes but in order to understand them, the intrusion process can be broken down into following four phasesPreprocessingAnalysisResponseRefinement1. Pre-ProcessingPreprocessing is the key function once the data is c ollected from IDPS sensor. The data is organized in some fashion for categorisation. The preprocessing helps in determining the format the data are put into, which is usually some canonical format or could be a structured database. Once the data are formatted, they are broken down further into classifications.These classifications can depend on the analysis schemes being utilise. For example, if rule-based detection is being used, the classification will involve rules and patterns descriptors. If anomaly detection is used, then statistical profile based on different algorithms in which the user behavior is baseline over the time and any behavior that falls outside of that classification is flagged as an anomaly.Upon completion of the classification process, the data is concatenated and put into a defined version or detection template of some object by replacing variables with values. These detection templates populate the knowledgebase which are stored in the core analysis engine. 2. AnalysisOnce the processing is completed, the analysis stage begins. The data record is compared to the knowledge base, and the data record will either be logged as an intrusion event or it will be dropped. Then the next data record is analyzed. The next phase is response.3. ResponseOnce information is logged as an intrusion, a response is initiated. The inline sensor can provide real time prevention through an automated response. Response is specific to the nature of the intrusion or the different analysis schemes used. The response can be set to be automatically performed or it can be done manually subsequently someone has manually analyzed the situation.4. RefinementThe final phase is the refinement stage. This is where the fine tuning of the system is done, based on the previous usage and detected intrusions. This gives the opportunity to knock down false-positive levels and to have a more accurate security tool.Analysis Process By Different Detection MethodsThe intrusion a nalysis process is solely depends on the detection method being used. Following is the information regarding the four phases of intrusion analysis by different detection methodsAnalysis Process By Rule-Based DetectionRule-based detection, also known as signature detection, pattern matching and misuse detection. Rule-based detection uses pattern matching to detect known attack patterns. The four phases of intrusion analysis process applied in rule-based detection system are as underPreprocessing The data is collected about the intrusions, vulnerabilities and attacks and then it is putted down into classification scheme or pattern descriptors. From the classification scheme a behavior model is built and then into a common formatSignature summon The given name of the signatureSignature ID The unique ID for the signatureSignature Description The description of the signature what it doesPossible False Positive Description An story of any false positives that may appear to be an exploi t but are actually normal network activity.Related Vulnerability Information This celestial sphere has any related vulnerability informationThe pattern descriptors are typically either content-based signatures, which examine the payload and header of packet, or context-based signatures that evaluate only the packet headers to identify an alert. The pattern descriptors can be atomic (single) or composite (multiple) descriptors. Atomic descriptor requires only one packet to be inspected to identify an alert, while composite descriptor requires multiple packets to be inspected to identify an alert. The pattern descriptors are then put into a knowledge base that contains the criteria for analysis.Analysis The event data are formatted and compared against the knowledge base by using pattern-matching analysis engine. The analysis engine looks for defined patterns that are known as attacks.Response If the event matches the pattern of an attack, the analysis engine sends an alert. If the e vent is partial match, the next event is examined. Partial matches can only be analyzed with a stateful detector, which has the ability to maintain state, as many IDS systems do. Different responses can be returned depending on the specific event records.Refinement Refinement of pattern-matching analysis comes down to updating signatures, because an IDS is only as sober as its signature update.Analysis Process By Profile-Based Detection (Anomaly Detection)An anomaly is something that is different from the norm or that cannot be easily classified. Anomaly detection, also referred to as Profile-based detection, creates a profile system that flags any events that strays from a normal pattern and passes this information on to output routines. The analysis process by profile-based detection is as followingPreprocessing The first step in the analysis process is collecting the data in which behavior considered normal on the network is baselined over a period of time. The data are put into a numeric form and then formatted. Then the information is classified into a statistical profile that is based on different algorithms is the knowledge base.Analysis The event data are typically reduced to a profile vector, which is then compared to the knowledge base. The contents of the profile vector are compared to a historical record for that particular user, and any data that fall outside of the baseline of normal activity is labeled as deviation.Response At this point, a response can be triggered either automatically or manually.Refinement The profile vector history is typically deleted after a specific time. In addition, different weighting systems can be used to add more weight to recent behavior than past behaviors.Section 3 IDPS TechnologiesThis section provides an overview of different technologies. It covers the major components, architecture, detection methodologies security capabilities of IDPS.ComponentsFollowing are the major components and architecture of IDPSSen sor Agents Sensors Agents monitors and analyze the network traffic for malicious traffic.SensorThe technologies that use sensors are network based intrusion detection prevention systems, wireless based intrusion detection prevention systems and network behavior analysis systems.Agents The term Agent is used for Host-Based Intrusion detection prevention technologies.Database Server The information recorded by the sensors and agents are kept safely in a database server.Console A console is software that provides an interface for the IDPS users. Console software is installed on the administrators PC. Consoles are used for configuring, monitoring, updating and analyzing the sensors or agents.Management Server It is a centralized device, receives information from sensors agents and manages that information. Some heed server can also perform analysis on the information provided by sensor agents, for example correlation of events. Management server can be both appliance based or so ftware based.3.1 Network architectureIDPS components are usually connected with each other through organizations network or through Management network. If they are connected through management network, each agent or sensor has additional interface known as management Interface that connects it to the management network. IDPS cannot pass any traffic between management interface and its network interface for security reasons. The components of an IDPS i.e. consoles and database servers are attached only with the Management network. The main advantage of this type of architecture is to hide its initiation from hackers intruders and ensure it has enough bandwidth to function under DoS attacksAnother way to conceal the information communication is to create a separate VLAN for its communication with the management. This type of architecture doesnt provide a much protection as the management network does.3.2 Security capabilitiesIDPS provide different security capabilities. Common secu rity capabilities are information gathering, logging, detection and prevention.3.2.1 Information gatheringSome IDPS gather general characteristics of a network, for example, information of hosts and network. They identify the hosts, operating system and application they use, from observed activity.3.2.2 Logging capabilitiesWhen a malicious activity is detected by the IDPS, it performs logging. Logs contain date time, event type, rating and prevention action if performed. This data is helpful in investigating the incident. Some network-based IDPS captures packet while host-based IDPS records user ID. IDPS technologies allow log to be store locally and send copies of centralized logging server i.e. syslog.3.2.3 Detection capabilitiesThe main responsibility of an IDPS is to detect malicious activity. Most IDPS uses combination of detection techniques. The accuracy and types of events they detect greatly depends on the type of IDPS. IDPS gives great results once they are properly tuned . Tuning gives more accuracy, detection and prevention. Following are some the tuning capabilitiesThresholds It is a value that sets the limit for normal and abnormal behavior. For example, the number of supreme login attempts. If the attempts exceed the limit then it is considered to be anomalous.Blacklists Whitelists A blacklist is list which contains TCP or UDP port numbers, users, applications, files extensions etc that is associated with malicious activity. A whitelist is a list of trenchant entities that are known to be benign. Mainly used to reduce false positive.Alert Setting It enables IDPS to suppress alerts if an attacker generates too much alerts in a short time and blocking all future traffic from that host. Suppressing of alerts provide IDPS from being overwhelmed.3.2.4 Prevention CapabilitiesIDPS offers multiple prevention capabilities. The prevention capability can be configured for each type of alert. Depending on the type of IDPS, some IDPS sensors are more inte lligent. They have learning simulation mode which enables them to know when an action should be performed-reducing the risk of blocking benign activity.3.2.5 Types of AlarmsWhen IDPS detects an intrusion it generates some types of alarms but no IDPS generates 100% true alarm. An IDPS can generate alarm for legitimate activity and can be failed to alarm when an actual attack occurs. These alarms can be categorized asFalse Alarms When an IDPS fails to accurately indicate what is actually happening in the network, it generates false alarms. False alarm fall into two main categoriesFalse Positives These are the most common type of alarms. False positive occurs when an IDPS generates alarm based on normal network activity.False Negatives When an IDPS fails to generate an alarm for intrusion, it is called false negative. It happens when IDPS is programmed to detect ck but the attack went undetected.2. square(a) Alarms When an IDPS accurately indicates what is actually happening in the n etwork, it generates true alarms. True alarms fall into two main categoriesTrue Positives When an IDPS detects an intrusion and sends alarm rigly in response to actually detecting the attack in the traffic. True positive is opposite of false negative.True Negative It represents a situation in which an IDPS signature does not send alarm when it is examining normal user traffic. This is the correct behavior.ARCHITECTURE DESIGHNArchitecture design is of vital importance for the proper implementation of an IDPS. The considerations include the followingThe location of sensors or agents.The reliability of the solutions the measurements to achieve that reliability. For example using of multiple sensors, for monitoring the same activity, as a backup.The number location of other components of IDPS for usability, redundancy and load balancing.The systems with which IDPS needs interfacing, includingSystem to which it provides the data i.e. log servers, management softwares.System to which i t initiates the prevention responses i.e. routers, firewalls or switches.The systems used to manage the IDPS components i.e. network management software.The protection of IDPS communications on the standard network.3.3 Maintenance OperationMostly IDPS are operated maintained by user graphic interface called Console. It allows administrator to configure and update the sensors and servers as well as monitor their status. Console also allows users to monitor and analyze IDPS data and generate reports. Separate accounts could be setup for administrators and users.Command Line Interface (CLI) is also used by some IDPS products. CLI is used for local administration but it can be used for remote access through encrypted tunnel.3.3.1 Common Use of ConsolesMany consoles offer drill down facilities for example, if an IDPS generates an alert, it gives more detail information in layers. It also give extensive information to the user i.e. packet captures and related alerts.Reporting is an impo rtant function of console. exploiter can configured the console to send reports at set time. Reports can be transferred or emailed to appropriate user or host. Users can obtain and customized reports concord to their needs.3.3.2 Acquiring applying updatesThere are two types of updates software updates and signature updates. Software updates for enhancing the performance or functionality and fixing the bugs in IDPS while the signature updates for adding detection capabilities or refining existing capabilities.Software updates are not limited for any special component but it could include all or one of them i.e. sensor, console, server and agents. Mostly updates are available from the vendors web site.New ChapterDetection MethodologiesMost IDPS uses multiple detection methodologies for broad accurate detection of threats but following are primary detection methodologiesSignature Based DetectionAnomaly Based DetectionStateful Protocol Analysis3.3.1 Signature Based DetectionThe term Signature refers to the pattern that corresponds to a known threat. In signature based detection, the predefined signatures, stored in a database, are compared with the network traffic for series of bytes or packet sequence known to be malicious, for example, an email with the subject of free screen savers and an attachment of screensavers.exe, which are characteristics of known form of malware Or a telnet