2018-11-28 Paul Johnson * ../R/importQualtrics.R (importQualtrics): dropText argument used 2018-11-27 Paul Johnson * ../R/variableKey.R (makeOneWide): (unique.one): if != 1, not > 1 (keysPool): add factor, character in legal class conversions (makeOneWide): Trying this approach now sep_old <- if("ordered" %in% unique(x$class_old)) "<" else "|" sep_new <- if("ordered" %in% unique(x$class_new)) "<" else "|" 2018-05-31 Paul Johnson * ../R/variableKey.R (makeOneWide): fix warning where x$missings and x$recodes did not exist in long2wide. Silences warnings in keyTemplateSPSS and keyTemplateStata. 2018-05-28 Paul Johnson * ../R/semTable.R (markupConvert): use file should not include ".tex", ".html" or ".csv" on end, but if it does, don't add another (markupConvert): table declaration now has 2018-05-23 Paul Johnson * ../R/variableKey.R (keyImport): new key column name check and cleanup. User column name errors are fixed by partial matching where possible. Will stop if name_old, class_old, or value_old columns are not present after that. Will try to fill in guesses for name_new, class_new, and value_new. If recodes and missings are not present, they will be created as columns of empty. 2018-05-18 Paul Johnson * ../R/modelcomparison.R (print.kutable): class "kutable" created, prevent unsightly output from compareLavaan and semTable. 2018-05-13 Paul Johnson * ../R/semTable.R (testtable): -batchmode and escape file name (semTable): redesign sequence to delay markup application. Planning for plain text output, alternative output mechanism. 2018-05-11 Paul Johnson * ../R/variableKey.R (keyTemplateStata): New function, same purpose as keyTemplateSPSS (statdatKey): does work for keyTemplateStata and keyTemplateSPSS 2018-05-09 Paul Johnson * ../R/semTable.R (semTable): obliterate underscores in variable names and variable labels. escaping in general does not work because of "pasted together" markup structure. Must rethink. * ../R/utils.R (modifyVector): error in replacement of named items discovered and corrected. 2018-05-01 Paul Johnson * ../R/variableKey.R (keyTemplate): remove "RECOMPILE" flag in keyTemplate example code (getUnique): in keyTemplate, trying to pass through unfamiliar classes with value_old = "." and value_new = "." 2018-04-17 Paul Johnson * ../R/factors.R (reverse): grep needed fixed = TRUE, else literal "." was matching everything. Users cannot pass in regular expressions for eol. * ../R/variableKey.R (keyTemplateSPSS): export this function. Fix sorting of variables. They did not come out in same order as SPSS. Only way I could think of: turn key wide, then order rows. 2018-03-27 Paul Johnson * ../R/semTable.R (loadingMaker): varLabels for table titles loadings/slopes. * ../R/semTable.R: bottom notes now nested in multicolumn at bottom. * ../R/utils.R (deduper): clarify description (modifyVector): cause repeat values in input to all be replaced, not just first. Uses replace function similar to mapvalues * ../R/semTable.R (parTableMaker): replace function created, don't use mapvalues to fix varLabels because it is annoying 2018-03-26 Paul Johnson * ../R/semTable.R (markupConvert): _STAR0_ trick to manipulate lineup. * ../R/semTable.R: label, caption, table, longtable fixes 2018-01-02 Paul Johnson * ../R/peek.R (projects, semTable): eliminate setwd and device creator 2017-12-27 Paul Johnson * ../R/mergeCheck.R: new function for inspecting merges * ../R/importQualtrics.R (importQualtrics): add XLSX file handling 2017-12-20 Paul Johnson * ../R/importQualtrix.R (importQualtrics): new function to ease qualtrix imports. * ../R/anonomize.R: new function to de-identify name columns * ../R/utils.R (getFiles): remove getFiles, which was copied from crmda package. 2017-12-14 Paul Johnson * ../R/likert.R (likert): return the percentages in the list as floating point numbers, not as characters. 2017-12-04 Paul Johnson * unitTests/runit.VariableKey.R (test.keysClassFix): unitTests on keysClassFix fail if class_new or class_old is FALSE, whole function does not make sense if we make that restriction. * ../R/variableKey.R (keyImport): factor separator now only "|", no longer either "|" or "<". For ordered variables, both still allowed. (cleanDataFrame): trimws on character and factor levels, otherwise key must include " Strongly Disagree" and "Strongly Disagree". The trimws argument has to be added to keyTemplate, keyImport, and keyApply. Default will be "both". * ../R/likert.R (likert): add ... argument, then deal with exclude and useNA in function. Not 100% working because useNA="ifany" is not correctly managed. 2017-12-01 Paul Johnson * ../R/likert.R (print.likert): likert returns now a list, this prints first item. 2017-11-30 Paul Johnson * ../R/variableKey.R (keySave): varlab corrected for CSV files 2017-10-23 Paul Johnson * ../R/semTable.R (semTable): trow is now just results, does not embed markup. Except for Row 1, which needs fixing. 2017-10-21 Paul Johnson * ../R/semTable.R (frnd): frnd: new formatC(round wrapper used to shorten code, replace all longer uses of format(round. (observedMeanMaker): abbreviate trows stanzas (roundSubtable): a common series of roundings is extracted from the maker functions. 2017-11-16 Paul Johnson * ../R/semTable.R (buildParamSetSubtable): if NA in merged table, change to "-" (markupConvert): change indentation in column 1 to right, other columns to centered. 2017-11-13 Paul Johnson * ../R/semTable.R: varLabels works, tested in examples 2017-11-12 Paul Johnson * ../R/utils.R (modifyVector): Clarify documentation, alter so that if y is NULL, then return x unaltered. 2017-11-10 Paul Johnson * ../R/semTable.R (testview): New function to compile and show tex tables. (buildSubtableForParamSet): if fits=NULL, then make sure it does not print. (buildSubtableForParamSet): working on group labels problem 2017-11-09 Paul Johnson * ../R/semTable.R (insertParamTypes): constructed parameters, cut out most of fixParamSets, omit constrained rows 2017-11-06 Paul Johnson * ../R/utils.R (starsig): correct "character(0)" return when no stars were called for! (starsig): if pval is a vector, but not numeric, us as.numeric. protects against columns set "as character" by formatC 2017-11-05 Paul Johnson * ../R/semTable.R (buildSubtableForParamSet): Lots of pussy-footing about changing the way labels are supplied and information is passed among vectors. (createEstSE): fix parameter rsquare for multi group * ../R/utils.R (modifyVector): new utility vector to replace named vector value selectively, also some other special feature shown in examples. 2017-11-03 Paul Johnson * ../R/semTable.R (extractModelSummary): multi-group fixes, treat 2 groups as 2 objects, rownames must be unique by group (createEstSE): insert group.labels as new column, makes multi-group work easier lalter on 2017-11-02 Paul Johnson * ../R/semTable.R (extractParameters): error b/c colNames is list, not vector anymore. (markupConvert): \tabularnewline should always insert \n with it. Results in "double spaced" tables, not problem in LaTeX output, need to see if that is problem in other output formats. (loadingMaker): fix the factor gotcha, where a character is converted to a factor and R re-alphabetizes it. (markupConvert): HTML markup fiddle, _BOC_BOM becomes "_BOM". silly fiddle. 2017-11-01 Paul Johnson * ../R/semTable.R (loadingMaker): now works with "slopes" and "loadings" parameter sets, deleting slopeMaker (parTableMaker): one table maker function, replaces interceptMaker, observedMeanMaker, thresholdMaker, residualvarianceMaker, residualcovarianceMaker, latentvarianceMaker, latentcovarianceMaker, latentMeanMaker (convertMarkupo): DANGER, _BOMC_ definition changed, now does not have & at front. DANGER, DANGER. 2017-10-30 Paul Johnson * ../R/semTable.R (cleanParamSets): Reduce this: if (length(which(parameters$rhs %in% latents & parameters$lhs %in% latents & parameters$op == "~~")) > 0) to if(any(params ...)) 2017-10-27 Paul Johnson * ../R/semTable.R (extractParameters): add trows class and trowsList class to mark the items within the parameter tables. 2017-10-26 Paul Johnson * ../R/semTable.R (getParamTable): new function to create parameters object, call rounding function here, no need within each stanza rename "cols" as "colNames", introduce "est(se)" as a legal column name. (parseParamTable): variables and latents cannot be global variables, must be attributes of parameter table, in order to allow for more than one model in this function (parseParamTable): handles params here, each model can have separate (latentMeanMaker): drop "gotcha" hit me again, chosing 1 column from trows 2017-10-25 Paul Johnson * ../R/semTable.R (markupTable): insert parameters as named argument in the maker functions >>>>>>> cleanupSemTable.R 2017-10-06 Paul Johnson * ../R/variableKey.R (keyApply): mapvalues insert warn_missing = FALSE on factor changes. (keyDiff): drop = FALSE inside row chooser, avoiding drop gotcha (makeOneVar): in makeKeyList, warnings about coercion of "." to integer were causing errors, so silence them by rephrasing coercion. (makeKeylist): Do not set NA's strings as ".", but rather R NA symbol. Hoping to quiet coercion errors & warnings. * ../R/runit.VariableKey.R: test.long2wide, test.wide2long, and keyUpdate. keyUpdate has hard coded "right answers" :( 2017-10-05 Paul Johnson * ../R/variableKey.R (keySave): rename "smartSave" -> "keySave" and "smartRead" -> "keyRead". Export both functions. (sortStanza): new function (naLast): new function. Put values that are missing last. (sortStanza): Drop Gotcha Again! How many times... 2017-10-04 Paul Johnson * ../R/variableKey.R (checkCoercion): new function, implements idea previously sketched as checkValue_new (long2wide): na.strings and missSymbol args added, change behavior of value_new when full new_value column is empty. No longer creates ".|.|.|", now just "." (isNA): new function, regular expression matching, for convenience of data checking. * ../R/variableKey.R (keyTemplate): re-trace data creation, include RECOMPILE to flag to re-generate package files. * ../R/variableKey.R (makeKeylist): completely worked over, avoids duplication of effort using wide2long(). * ../R/variableKey.R (keyUpdate): rework 2017-09-29 Paul Johnson * ../R/variableKey.R (keyImport): replace if (wide) code with wide2long, then trust the long key work. Cuts risk of error in half. 2017-09-28 Paul Johnson * ../R/variableKey.R (shortenValues): in keyTemplate, was error that caused redundant key lines. Hopefully corrected by unique(xismissing) (assignMissing): add treatment of logical variables. * unitTests/runit.VariableKey.R (## test.keyDiff): testing keyDiff fails now because i've restructured keyDiff output, so comment this out. * ../R/variableKey.R (keyDiff): recode around making a diff structure composed of 2 data frames: "deleted" is lines removed from oldkey and "neworaltered" are lines in newkey that were not present in oldkey. 2017-09-27 Paul Johnson * ../R/variableKey.R (keyDiff): print message to user should be an attribute. output is a list of 2 data frames, one for deleted lines from the old key and one for added or altered lines in new key. * unitTests/runit.VariableKey.R (test.keyClassFix): hard-coded values needed revision to match current format of keys. * ../R/variableKey.R (smartSave): Using workbook was probably unnecessary, but does work now. Problem about finding sheet names solved by "getWorkspaceNames" function in openxlsx. (keyUpdate): Replace this line-dependent code, which caused example to fail when data failed: key1.long[c(5, 6, 7), "value_new"] <- c(10, "female", "male") 2017-09-26 Paul Johnson * ../R/variableKey.R (assignMissing): if missings=NULL, return x. (smartSave): na.strings new argument, "." default, will only affect long keys (smartRead): na.strings new argument, "." or space will be R symbol NA. (keyTemplate): delete the outdir argument. Was dumb! (smartSave): use Excel Workbook. 2017-10-24 Paul Johnson * semTable.R (markupTable): Insert EOC BOC EOR syntax into numeric table 2017-09-25 Paul Johnson * ../R/variableKey.R (keyLookup): new function to find name_new or name_old values in key (keyTemplateSPSS): new function to build keys for SPSS data transitions 2017-09-21 Paul Johnson * ../R/utils.R (getFiles): file copy wrapper for theme files, identical to same in crmda/utils 2017-09-19 Paul Johnson * ../R/utils.R (writeCSV): new function to create CSV files with same quotation marks as Excel (writeCSV): tidy up csv commentary * ../R/variableKey.R (all.equal.keylong): all.equal methods for keys, handle sorting of rows to make data.frames identical, does not check attributes to avoid errors about row name and varlab differences. (varlabTemplate): varlab is now a named vector attribute. Stored on disk differently, see smartRead and smartSave. 2017-09-18 Paul Johnson * ../R/variableKey.R (smartRead): tryCatch failed when varlab was not a sheet. 2017-09-11 Paul Johnson * ../R/variableKey.R (varlabTemplate): correct Roxygen 2017-07-11 Paul Johnson * ../R/utils.R (deduper): new function to clean strings * ../R/variableKey.R (keyImport): Logicals: change value_new from 1/0 to TRUE/FALSE. This fixes a problem caused by LibreOffice storing TRUE/FALSE as 1/0. 2017-07-10 Paul Johnson * ../R/variableKey.R (keyDiagnostic): nametrunc-fiddle, make name old longer, change keyDiag logic on name_new missing. 2017-06-27 Paul Johnson * ../R/variableKey.R (keyApply): fix error in creation of factors because of numeric -> factor error (wrong levels) 2017-06-21 Paul Johnson * ../R/variableKey.R: new functions keyCheck, keysPoolCheck, keyClassfix. 2017-02-03 Paul Johnson * ../R/variableKey.R (assignMissing): zapspace for spaces 2017-02-01 Paul Johnson * ../R/variableKey.R (assignMissing): significant revamp of notation for missings "8;9;(30,40);>50" 2017-01-31 Paul Johnson * ../R/variableKey.R (smartRead): insert for loop to check column classes of key import via read.xlsx. Any column that is not characters is re-set as character. (long2wide): printvals function was inadvertently changing all missing values in value_new to "". We should instead have either NA for missings, but preserve "NA" as a valid string if it is not marked in the na.strings variable of keyImport. (makeKeylist): replace "." with NA values in value_new (printvals): long2wide: All missing values in character variables become "." symbols. (assignMissing): missings string needs to protect us against ">-999" "<-100" by insering space. 2017-01-25 Paul Johnson * ../R/cfaTable.R (cfaTable): clean up example so that no file is created. Output is tex code. 2017-01-09 Paul Johnson * ../R/modelcomparison.R (compareCFA): replace for loop with mgsub, before that tighten up "name_old" to be always full string "^chisq$", not just "chisq". This eliminates accidental matches. Wish I could limit that to only the header row, but it happens after xtable has done its work. * ../R/cfaTable.R (markup): cat(template) then return invisible(template) 2017-01-03 Paul Johnson * ../R/updatePkgDepends.R: do not specify default repos as KU-specific preferences. * ../DESCRIPTION: depends on R >= 3.3.0 2016-12-22 Paul Johnson * ../R/likert.R: example with file output is commented out * ../R/peek.R: examples: No file output * ../R/cfaTable.R (cfaTable): examples \donttest on file output examples. 2016-12-15 Paul Johnson * ../R/variableKey.R (keyDiagnostic): clean up message in diagnostic for variables with too many values. (long2wide): insert comment (keyDiagnostic): clarify comment to user about wide editor display 2016-12-07 Paul Johnson * ../R/utils.R: dts had no documentation, so it was inserted. * ../R/project.R (initProject): Remove hard coded directory names, rely on arguments to set them. Insert new argument gitArgs for git init customization. Clear up documentation, revise examples to be more informative. 2016-12-02 Paul Johnson * ../R/dir.create.unique.R (dts): imported basics from rockchalk package, fixed directory naming error if date = TRUE, clarified error handling. 2016-11-10 Paul Johnson * ../R/variableKey.R (keyApply): fix error B&B found in !class 2016-11-08 Paul Johnson * ../R/variableKey.R (keyApply): create new if/then for factor -> numeric conversion. (printvals): Brent's example showed that value_old as "NA" was manufactured by long to wide in a paste statement. ifelse logic should prevent that now. 2016-11-07 Paul Johnson * ../R/variableKey.R (keyDiagnostic): if variable is not in data, do not run key diagnostic. * ../R/peek.R (dev.create): create graphic device on user workstation. Needed to defeat R Studio * ../R/variableKey.R (cleanDataFrame): use new function is.data.frame.simple. (is.data.frame.simple): new function returns FALSE if data frame is not made up of columns. 2016-10-28 Paul Johnson * ../R/variableKey.R (checkValues): prevent NA from triggering output (keyApply): add debug argument 2016-10-25 Paul Johnson * ../R/textUtils.R (escape): html escape "/" as well (alphaOnly): insert "also" argument 2016-10-24 Paul Johnson * ../R/variableKey.R (## keyUpdate):demonstrate mydf case after adding Jan to new data 2016-10-20 Paul Johnson * ../R/updatePkgDepends.R (updatePackages): pass through arguments ask and checkBuilt from command line to update.packages. 2016-10-09 Paul Johnson * ../R/project.R (initProject): insert new rheader file which defines usual directories, named "rheader.R" in the R folder 2016-10-06 Paul Johnson * ../R/variableKey.R (keyTemplate): Changed naming so that key templates are suffixed by "templ" and the edited keys will be "key". No more "key_new" or "keylong_new", but now just "key" and "key_long". (keyUpdate): rework handling of new variables and new values, relabel argument from bottom to append, new example stress tests. (keyApply): was accidentally allowing stringsAsFactors=TRUE, which prevented some variable class assignments. 2016-09-23 Paul Johnson * ../R/utils.R (padW0): fix error that was causing integers to turn to all NA. (stringbreak): vectorize stringbreak. 2016-09-22 Paul Johnson * ../R/variableKey.R (wide2long): rework to correct handling of recodes and missings, which should not be repeated throughout the column when there is only one value. 2016-09-21 Paul Johnson * ../R/variableKey.R (long2wide): correct problem that NA values were turned to "NA" strings in missings and recodes. 2016-09-20 Paul Johnson * ../R/variableKey.R (keyApply): The use of split was causing the name_old.new values to be alphabetized. We don't want that. (keyUpdate): 1) T -> TRUE. 2) replace "wide" with "long" for consistency with other functions 3) Don't do wide == TRUE, recommended to do isTRUE(wide). Inserted comments to suggest change in design. (long2wide): create name_old.new and set as factor with ordering from original key, trying to prevent re-alphabetization. (wide2long): insert sep argument, use while splitting key input. (keyImport): Rip apart keyImport, redesign as key importer and cleaner. This should handle all cleanup of user input and make sure column names are correct. Does NOT return keylist object, make separate function for that. (makeKeylist): Does work previously by keyImport. Only called from inside keyApply, so as to simpify user experience. 2016-09-09 Paul Johnson * ../R/variableKey.R (cleanDataFrame): new function that applies safeInteger and checks that the data frame is made of columns, no matrices embedded. (keyApply): Ignore capitalization of "name_old" in scanning for columns in data frame. (smartSave): new function that does file save (keyStacker): pile up key templates 2016-09-08 Paul Johnson * ../R/variableKey.R (safeInteger): new function to coerce numerics that are "close enough" to integers as integers. (keyTemplate): new argument safeNumericToInteger, uses safeInteger to set classes on columns in data frame being recoded. (keyDiagnostic): conditionalize the confidentiality protecting rounder argument. (wide2long): Correct example, it verifies now! (long2wide): Correct example, it verifies now! 2016-09-03 Paul Johnson * ../R/utils.R (padW0): front-pad values with zeroes. 2016-09-01 Paul Johnson * ../R/variableKey.R (wide2long): new utility key converter function. (long2wide): new utility key converter function. 2016-08-30 Paul Johnson * ../R/variableKey.R (keyApply): use assignRecode in for loop, so each element in the recodes is applied in a sequence. 2016-08-29 Paul Johnson * ../R/variableKey.R (assignRecodes): new function to apply the key 2016-08-22 Paul Johnson * ../R/variableKey.R (keyDiagnostic): new function to compare old and new data against variable key (keyApply): Dealing with duplicated factor levels problem, avoiding them, reassinging values correctly. If $v$value_old is empty, what to do? Too risky to assing value_new in that case, cutting out code which tried to guess and do so. 2016-08-21 Paul Johnson * ../R/variableKey.R (keyImport): Redesign to allow multiple name_new for a single name_old. names on keylist object are "name_old.name_new" for clarity. 2016-08-19 Paul Johnson * ../R/variableKey.R (getValues): max.levels change. If integer or character is over the limit, then just print one line in key, just as we do for real number variables. * vignette/variablekey.lyx: revise commentary about keyTemplate to match new interpretation of max.values. 2016-08-17 Paul Johnson * ../R/utils.R (stringbreak): New function to insert "\n" into strings * ../R/peek.R (quickbar): barplot error because some category values too ridiculously long. Now implement LIMIT, will later insert line breaks * ../R/utils.R (shorten): add shorten funtion with unique 2016-08-12 Paul Johnson * ../R/modelcomparison.R (compareCFA): insert @export and @importFrom * ../R/cfaTable.R (cfaTable): fitMeasures -> lavaan::fitMeasures * ../R/modelcomparison.R (compareCFA): fitMeasures -> lavaan::fitMeasures (compareCFA): break long lines in stop with paste() 2016-08-11 Paul Johnson * ../R/assignMissing.R (getUnique): pack the keyTemplateLong and keyTemplate functions into one single function with the argument long = TRUE/FALSE to differentiate them. * ../R/project.R (initProject): output "there is no git" message. 2016-08-11 Paul Johnson * ../R/assignMissing.R (recodeDF): first working version 2016-08-10 Paul Johnson * ../R/assignMissing.R (cleanDF): rough in work to apply a keylist object to a data frame. 2016-08-09 Paul Johnson * ../R/assignMissing.R (n2NA): n2NA accepts more arguments for types of values to be removed; zapspace argument. * ../R/assignMissing.R (zapspace): remove leading and trailing spaces. 2016-08-08 Paul Johnson * ../R/assignMissing.R (unique2): In keyimport, chose plan to create an output list. * ../R/assignMissing.R (keyTemplateLong): make strings, not factors in new key. 2016-08-03 Paul Johnson * ../R/assignMissing.R (keyTemplateLong): remove usage of openxlsx in function, put as example, add Suggests to DESCRIPTION document. 2016-08-02 Paul Johnson * ../R/assignMissing.R (assignMissing): overhaul with documentation to deal with different inputs for doubles, integers, factors * ../R/project.R (initProject): Sys.which("git") to check if user's system has git in path. Could we search more aggressively if it exists elsewhere? * ../R/deleteBogusRows.R (deleteBogusRows): add arguments drop, verbose, and n, use NCOL. Argument n allows NULL or NA values. (deleteBogusColumns): a column version of deleteBogusRows. Should have planned ahead and written one function with a margin argument, may come back later work on that. 2016-08-01 Paul Johnson * ../R/peek.R (varType): if noneoftheabove variables are not present, do not print a warning about having to exclude them. Rocket Science Alert. * ../R/utils.R (floorvar): delete. I cannot think of any reasonable use case for this. (starsig): insert new argument symbols to allow users to have whatever symbols they want. * ../R/peek.R (varType): revise to allow single columns in addition to data frames 2016-07-31 Paul Johnson * ../R/project.R (initProject): new function to create project directories, starts git repo, adds readme and changelog files. 2016-07-29 Paul Johnson * ../R/histOMatic.R: fix documentation about arguments, reformat as details. 2016-07-28 Paul Johnson * ../R/histOMatic.R (quickbar): histargs, barargs inserted, using to reduce warnings due to arguments that do not fit. * ../R/utils.R (removeMatches): new function, adapted from Ben Kite code (mgsub): multi-gsub function, adapted from loose recollection of something Jared harpole wrote. * ../R/histOMatic.R (varType): dots arguments for pdf are now extracted. Corrected ask logic when file is true. New, richer examples showing combinations. 2016-07-27 Paul Johnson * ../R/histOMatic.R: change histOMatic to "look", handles numeric, string, or factor variables, can output on screen, in file, also option to put text in session. * ../R/updatePkgDepends.R: Accelerate by vectorization, insert BIOC as repo, clean up docs 2016-05-23 Paul Johnson * ../R/utils.R (stars): The stars function conflicted with base package stars, so rename "starsig" 2016-03-08 Paul Johnson * TODO: Created a TODO file, and ChangeLog for first time