-- An example of text processing.
-- by Scot Drysdale, Sept. 2007.

-- Takes a text string (with punctuation, etc.), breaks it 
-- into words, and computes average word length
-- Could be done more briefly using built-in functions

-- Replaces newlines and tabs by spaces
whitespace2spaces :: [Char] -> [Char]
whitespace2spaces ('\n' : rest) = ' ' : whitespace2spaces rest
whitespace2spaces ('\t' : rest) = ' ' : whitespace2spaces rest
whitespace2spaces ('\r' : rest) = ' ' : whitespace2spaces rest
whitespace2spaces (ch : rest) = ch : whitespace2spaces rest
whitespace2spaces [] = []

-- Returns true if the character is a letter or a space
isLetterOrSpace :: Char -> Bool
isLetterOrSpace ch = elem ch 
  "abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ"

-- Breaks the input string into a list of words  
breakWords     :: [Char] -> [[Char]]
breakWords str =  breakHelper [] 
  (filter isLetterOrSpace (whitespace2spaces str))

-- Breaks the input string into words using an extra parameter.
-- The first parameter is the word accumulated so far, 
-- the second is the remainder of the input text.
-- Returns a list of words
-- This version is not as efficient as it could be
breakHelper                      :: [Char] -> [Char] -> [[Char]]
breakHelper [] []                = []
breakHelper word []              = [word]
breakHelper [] (' ' : rest)      = breakHelper [] rest
breakHelper word (' ' : rest)    = word : breakHelper [] rest
breakHelper word (letter : rest) = breakHelper (word ++ [letter]) rest

-- Computes the average word length of a piece of text.
averageWordLength     :: [Char] -> Double
averageWordLength str = totalLength / intToDouble (length wordList)
  where
    wordList = breakWords str
    totalLength = intToDouble (foldl (+) 0 (map length wordList))

-- Converts an Int to a Double
intToDouble   :: Int -> Double
intToDouble n = fromInteger (toInteger n)

newYorkTimes = "G.M. and Union Reach Tentative Agreement\n By MICHELINE MAYNARD and NICK BUNKLEY\nPublished: September 26, 2007\nThe United Automobile Workers union and General Motors reached a landmark agreement early today, ending a two-day strike.\nThe key provision of the new contract is a health care trust that would get G.M.’s vast liability off its books, while workers and retirees would receive their current coverage until the trust is put in place.\nThe deal was announced by the company and the union in separate statements. The U.A.W. had walked out on G.M. on Monday morning, but production will resume this afternoon.\nG.M. said the tentative agreement was reached at 3:05 a.m. Eastern time. The U.A.W. recessed the strike and said that if the contract was not ratified, workers could return to picket lines./nThe agreement included a memorandum of understanding to establish an independent health care trust, as well as other changes to the national agreement.\nG.M. said implementation of the trust would be subject to court approval, as well as a review by the Securities and Exchange Commission.\nThe memorandum establishes the principle of the trust, called a voluntary employee benefit association, or VEBA. It will be funded by cash, G.M. stock and other assets, people with direct knowledge of its components said today.\nG.M. and the U.A.W. do not expect that the trust will receive final approval until at least 2009.\nThe two sides agreed that the company could tap its pension fund, which is overfunded by several billion dollars, to reimburse retired workers for the increase in health care benefits that they have seen over the last two years, since the union made health care concessions to G.M.\nOnce the VEBA is established, G.M. and the U.A.W. would review it annually. If it is found to be underfunded, G.M. would make further contributions, people with direct knowledge said, although there could be limits on how much G.M. would add.\n“There’s no question this was one of the most complex and difficult bargaining sessions in the history of the G.M./U.A.W. relationship,” Rick Wagoner, G.M.’s chief executive, said in a statement./nGrant Muncy, chairman of Local 211 in Defiance, Ohio, said workers were “just hoping for a fair deal,” adding, “I hope that it means we’ve got jobs coming back to the U.S.A. for the American workers.”\nU.A.W. leaders are likely to meet on Friday to consider the contract. If approved, it would go to workers for a vote.\nThe union’s president, Ron Gettelfinger, said the new contract “will absolutely protect their jobs and keep jobs from being reduced.” He said, while not offering specifics, that the number of jobs at G.M. would be “pretty much the same if not higher” when the contract concludes in 2011.\nLater, Mr. Gettelfinger confirmed in a radio interview that there was a signing bonus for workers, but declined to state its size. He also declined comment on reports that the contract contained a two-tier wage program, with sharply lower rates for any new workers hired by G.M.\nThe U.A.W. had designated G.M. the lead company in its contract negotiations with the Detroit automakers, and the old agreement with the company expired at midnight on Sept. 14.\nRather than strike immediately, the union extended the contract on an hour-by-hour basis. But on Monday, the U.A.W. struck G.M. when it failed to reach agreement by a strike deadline of 11 a.m. Eastern time.\nThe union had not staged a national strike against G.M. since 1970; its last major walkout, at two plants in Flint, Mich., was in 1998.\nThe latest strike was the sufficient push the two sides needed to reach an agreement in the talks, which G.M. called the most important in a generation.\nG.M.’s key demand was the VEBA, which would include the union’s participation. G.M. would be the first among the Detroit companies to take full responsibility for coverage for active and retired workers and their families. G.M. estimates that liability at $55 billion.\nSimilar trusts could soon follow at the Ford Motor Company and Chrysler.\nThe automakers have pushed hard for the union to agree to form such trusts, maintaining that their so-called legacy costs hinder their ability to compete with Japanese auto companies, whose costs are lower.\nAll told, the three companies have health care liabilities totaling nearly $100 billion."

usaToday = "GM-UAW reach tentative deal; strike ends\nBy Sharon Silke Carty, USA TODAY\nDETROIT — General Motors (GM) and the United Auto Workers reached a historic agreement early Wednesday on a four-year contract that sets up a union-run trust fund for retiree health care./nUnion members were told to begin reporting to work Wednesday.\nSetting up the trust fund will improve GM's financial situation by getting retiree health care costs off its books. The move also likely will make the UAW one of the largest lobbying forces in the country on health care issues.\n\nSTATEMENTS: From GM and the UAW\n\nThe agreement, which must be approved by union members, came less than 48 hours after the UAW called a national strike against GM, pulling 74,000 workers off assembly lines. GM and the UAW negotiated 12 days past the end of the last contract.\nFIND MORE STORIES IN: Wednesday | General Motors | United Auto Workers | Detroit Free Press | GNS | Ron Gettelfinger | Mandi Wright\nIn addition to creating the health care fund, the deal would allow GM to hire new workers for lower wages and benefits. UAW members are eligible for a $3,000 bonus if they ratify the contract, but they give up raises and cost of living adjustments.\nThere's no question this was one of the most complex and difficult bargaining sessions in the history of the GM/UAW relationship, GM CEO Rick Wagoner said.\nWe're proud of this tentative agreement, and we look forward to getting into the field and discussing it with our membership, UAW President Ron Gettelfinger said.\nWorkers on the picket line this morning were relieved, but wanted information.\nIt sounds good that this might be over with, said Matt Cauley, 46, a 28-year GM employee picketing outside the Romulus, Mich., engine plant.\nI want to see the details, said Steven Cichone, who had just finished a shift picketing at GM's Hamtramck, Mich., assembly plant when the deal was announced. The 10-year GM employee said he would be pretty happy as long as there's no pay cut.\nThe UAW next will move to wrapping up contract negotiations with either Ford Motor (F) or Chrysler this week. Historically, the contract negotiated with the first automaker sets a pattern for contracts with the other two, although the UAW has not always given all three the same treatment. Chrysler was left in the cold two years ago after Ford and GM brokered deals that passed more health insurance costs to workers.\nGettelfinger indicated in a radio interview this morning that the union may conduct simultaneous talks with Ford or Chrysler next.\nGM went into the talks aiming to shave $25 off its total hourly compensation figure, which includes retirement health care, pensions and other benefits. Those costs are said to put U.S. manufacturers at a $1,500 to $2,000 disadvantage to the foreign automakers.\nGM said those benefits make the wages it pays its workers uncompetitive with Asian automakers.\nThis agreement helps us close the fundamental competitive gaps that exist in our business, Wagoner said. The projected competitive improvements in this agreement will allow us to maintain a strong manufacturing presence in the United States along with significant future investments.\nWall Street cheered the agreement, sending GM shares up 6% in Wednesday trading.\nWe view the tentative agreement as a historic milestone toward the long-term improvement in fundamentals and survival of the North American automakers, Brett Hoselton, KeyBanc auto industry analyst, wrote to investors."