ToolboxNew Products for IT Pros

Greg Steen

The opinions expressed in this column are solely those of the author and do not necessarily reflect the opinions of Microsoft. All prices were confirmed September 11, 2008, and are subject to change.

Map Data Graphically

MapForce

www.altova.com/products/mapforce/data_mapping.html

One of the perhaps not-so-joyous realities of the increasing number of inter- and intra-connections in today's application stacks is that you need to somehow move data between those applications. Straight one-to-one mappings between those applications are very rare, and so you'll often find yourself spending time designing some kind of ETL (extract, transform and load) process in order to get that information from one place to another in compatible formats. MapForce from Altova can help streamline that process and make it much easier.

This tool lets you graphically map out your data-transformation process with your mouse. Source and target data sources can be text or CSV files, XML, databases, Electronic Data Interchange (EDI) data, Microsoft Office Excel 2007 data, and even Web services. In terms of database source options, MapForce supports SQL Server, DB2, Oracle, Sybase, MySQL, and Microsoft Office Access, so you can connect to almost any database.

To get started, you create your data sources and targets and then map input fields to output fields. Of course, you may not want direct one-to-one or every-row-type mapping, so MapForce also provides a number of data-processing functions for filtering data. The tool allows you to build and use conditional statements based on Boolean logic, string operations, mathematical computations, and more to sort out the data.

The function library that comes with MapForce includes loads of ready-to-go functions to help you transform your data. For example, you can use math functions to pull minimum and maximum values, averages, and computed values. You can turn database NULL values into an empty string, convert Greenwich Mean Time to Eastern Standard Time, or match patterns within a string. In addition, MapForce allows you to add new function libraries as they are developed.

Once you have your mapping configured, you can immediately see the results by clicking on the Output tab. (This is handy for quickly making sure you have your data in the right place.)

The UI can take some getting used to. But MapForce has features in place to help with the adjustment, such as the Overview pane for complex transformation tasks, which gives you a zoomed-out view of where on the map you are looking. This is convenient for jumping between regions in the process.

Another nice feature of the latest edition is support for the Excel 2007 Office Open XML (OOXML) format as a direct data source; there's no need for an ODBC driver or similar component. In addition, Altova has added a great deal of support for Web services, letting you use Web methods as data sources, as targets, and even to transform your data. To take advantage of this feature, all you have to do is provide the Web Services Description Language (WSDL) definition for the service, its location, and its credentials. And the application is also capable of fully supporting both the SOAP 1.1 and SOAP 1.2 specifications.

With a database mapping, you can use the built-in query tool to build up a SQL statement to select the data in which you are interested; MapForce even has type-ahead completion for doing so. With this tool you also have the ability to define primary and foreign key constraints, as well as have the product do all the ID generation if you don't want the database to handle this.

As a developer, one of my favorite features of MapForce is the ability to generate a Visual Studio 2008 project with program code to execute your data transformation. And if you prefer to go from code to MapForce rather than from MapForce to code, you can download a plug-in for Visual Studio. In addition to generating code in C# for Visual Studio, you can generate code in Java and C++.

When it comes to purchasing the software, if you don't need the Web service mapping support, the FlexText dynamic text file parser, or support for OOXML, you may want to opt for the less-expensive Professional Edition. A further limited Standard Edition is also available.

Price: $1,190 for the Enterprise Edition; $599 for the Professional Edition; $299 for the Standard Edition.

Altova MapForce lets you map your data graphically (Click the image for a larger view)

Create and Manage Regular Expressions

Expresso

www.ultrapico.com/expresso.htm

Whether you're a developer, systems administrator, network specialist, or QA analyst, you have probably found yourself parsing some kind of log file looking for patterned information. Regular expressions can be a very powerful method for getting to the exact information you are looking for in those logs. And applications such as Windows PowerShell are very handy tools to rely on. But, with their sometimes seemingly esoteric and perhaps lengthy structures, these tools can make it extremely difficult to construct the expressions as you need them. If this sounds familiar, consider giving Expresso from Ultrapico a try. This tool makes it much easier to develop, test, and reuse regular expression.

Expresso's main UI provides three basic views for development: Test mode, Design mode, and the Expression Library view. In Test mode, there are four panes of functionality. The Regular Expression pane is where you actually edit the expression to find and/or replace the data you are looking for. The Sample Text pane shows the text that the regular expression will run on and to which you can paste in an appropriate sample. The Regex Analyzer pane gives you a very cool hierarchical tree view of your regular expression components, helping you see how each component of the expression will operate. And the Results pane, as you might expect, shows the results of the operations you perform.

Within Test mode, there are six operations you can perform with your expression to get those results: full match, partial match, match exclusion, replacement, validation, and split. Full match takes your regular expression and runs it against the sample text, while partial and exclusion matches take the part of the expression you highlight and match it with or exclude it from to ensure the inner parts of your expressions are working properly. And you can use the tree view to navigate and highlight a particular section automatically. Replacement and split do just as the names imply. And validation uses icons to indicate which lines in your sample text will match.

Design mode provides the Regex Analyzer and Regular Expression panes as well as a place for your replacement text (you will likely be flipping back and forth between the Test and Design modes as you work). Also, you get a great designer toolbox, which helps you build up complex expressions with simple GUI actions. You can insert blocks that work with characters, groups, position, repetitions, and substitutions; and find helpers for ASCII and Unicode character and control codes.

The Expression Library gives you a history of your project's expressions. The application is project based, so you can save your works in progress or frequently used workspaces and come back to them when you need to. A number of sample projects also come with the application to help you get started with regular expressions, such as a built-in overall Regular Expression Library function. The default library comes loaded with expressions you can tailor to your own needs. You can add (and delete) expressions or create your own libraries.

If you find yourself using the same few expressions repeatedly, just move them to the top of the order. Double-clicking an expression in the library dumps it back into the main application window so you can immediately apply it in your workspace.

Expresso also has built-in code generation functionality. This will create a C#, C++, or Visual Basic code wrapper for your designed expression set. And, if you want, it will even compile this into a ready-to-go DLL for use in your applications or Windows PowerShell scripts.

Whether you are just starting out with regular expressions, have been learning over time, or are a certified guru looking for a way to test and save your creations, check out Expresso. This free utility could become one of your most valuable tools.

Price: Free.

Expresso makes it easier to develop, test, and reuse regular expressions (Click the image for a larger view)

Query and Navigate Active Directory

Active Directory Query

www.tumbleson.net/products/products.htm

Querying and navigating through Active Directory are ongoing tasks for the systems administrator. Fortunately, there are tools available to help make those tasks a bit easier and more manageable. One tool I like is the easy-to-use Active Directory Query from Tumbleson.NET.

On startup, this application scans for domains within the forest to which the machine is connected and adds them to its internal list for reuse. You can also add domains manually, as well as provide alternate credentials so the application has access to query the domains in which you are interested.

Active Directory Query has a number of built-in queries and also supports custom queries. Built in, the application can query users, contacts, computers, groups, printers, shares, deleted objects, and recently created objects for the target organizational unit (OU).

After you've executed a basic query, you can then filter the results by enabled or disabled directory objects or by a string of text. In addition, there are computer filters that will show you stale computer accounts; and there are user filters that show locked accounts, expired accounts, stale accounts, expired passwords, and user accounts that have never actually logged into the domain.

For user accounts, you can easily enable and disable accounts, lock and unlock accounts, reset account passwords, display which Groups accounts are members of, and delete accounts. You can click to quickly contact a user (via e-mail, instant message, or phone). You can even upload a JPEG image to associate with an account.

When querying machines, Active Directory Query pulls back useful information, such as the OS, service pack, Fully Qualified Domain Name (FQDN), and LDAP path into an easy-to-scan table, letting you quickly get a view of the machines in question. There are a number of useful quick-click functions available here also—you can enable and disable a system, remote desktop to the system, provide remote assistance, perform management, power up and down a system, and get a list of the current shares on the computer. You can also trigger a ping, trace route, nslookup, and HTTP or FTP connection to the machine. And if you need more information, you can retrieve hardware, software, network, running process, and logged-in user information.

Another nice feature: you can save and create hotkeys for up to 10 custom queries so you won't have to craft them each time you need the information. Active Directory Query can also use wake-on-LAN (WOL) to remotely start machines that are powered down.

To do this, simply query for the machine or list of machines you want to wake up, and then right-click to retrieve the computer's MAC address via Windows Management Instrumentation (WMI). The application uses the MAC, which is stored in the application's internal database, to send a WOL packet to the machine.

If you need to, Active Directory Query can export to Excel, Notepad, or your clipboard with a couple of clicks, allowing you to save, share, or repurpose the information returned by your query. All in all, this is a useful and very easy-to-use tool.

Price: $89.99.

Query and manage systems with Active Directory Query (Click the image for a larger view)

Book Review

Crimeware: Understanding New Attacks and Defenses

www.symantec.com/business/solutions/symantecpress.jsp

Users are becoming more aware of both cybercrime and the need to protect information (both personal and corporate). Still, attacks are becoming more sophisticated and more widely distributed; they continue to be successful, be it by stealing credit cards, personal information, or corporate data, or even "misinforming" targets.

Whether you think you are directly involved in your organization's security policies or not, you can have a direct impact on them by becoming more aware of the latest tools and techniques used by attackers. One book to help you get up to speed is Crimeware: Understanding New Attacks and Defenses (Symantec Press, 2008), by authors and security professionals Markus Jakobsson and Zulfikar Ramzan.

So what is crimeware? Well, as the book succinctly states, the "goal is profit or political power," whereas the more familiar-termed malware strove primarily for "notoriety." As you'll see in the introduction, the techniques and toolsets at the disposal of the purveyors of crimeware are extensive: keyloggers, screen scrapers, Web Trojans, transaction generators, system reconfiguration, proxy attacks, rootkits, piggybacking, and the list goes on.

After discussing all those tools and techniques, the book shows how applications are ripe with opportunity for attack through what it calls the Trinity of Trouble: connectivity, complexity, and extensibility. It goes on to show numerous platforms that can be exploited and/or targeted, such as peer-to-peer networks, handheld and RFID devices, firmware, browsers, and bot networks. Within these sections, you'll learn things such as how cross-site scripting attacks function, how bot networks are propagated and utilized, and how WiFi routers can be infected with crimeware.

In the next section, you get a look at the rootkit and learn about the kernel-mode hooks that can be placed on an unsuspecting system and how a rootkit can redirect memory access or filter through file system changes while hiding itself from detection. There are also chapters on fraud within online virtual worlds and online advertising fraud, which discusses the wide array of technologies, techniques, and social-engineering strategies used by criminals to get your money and information. The chapter on business models shows just how profitable these attacks can be.

But, ultimately, the book provides some hope with a couple of sections covering the laws in place to protect against cybercrime and how trusted computing and other defense techniques can limit exposure to attack. If you are looking to get up to speed on the techniques of attack and defense in practice today, consider giving this book a good cover-to-cover read to understand current attacks from all angles.

Price: $49.50.

Greg Steen is a technology professional, entrepreneur, and enthusiast. He is always on the hunt for new tools to help make operations, QA, and development easier for the IT professional.