How to find or search for invalid characters for OneDrive upload

Looking how to find or search for the following invalid characters in all the shared folders to prepare to upload to OneDrive.  

The characters are:

\
/
:
*
?



|
#
%
LightWireAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

arnoldCommented:
What are you using to upload? a pattern match in your application to validate the file name .......
0
LightWireAuthor Commented:
I need to verify my data structure before I proceed in the upload.  Is there a script I can use to produce an list of the output so it can be modified?
0
arnoldCommented:
Yes.

Depending on what it is you are using, you have to look at the naming convention you use and make sure it matches the requirements .
0
Python 3 Fundamentals

This course will teach participants about installing and configuring Python, syntax, importing, statements, types, strings, booleans, files, lists, tuples, comprehensions, functions, and classes.

LightWireAuthor Commented:
I understand, but this is a customer's structure and they have over 100k + files/folders and I would like to search via a script instead of one at a time.
0
arnoldCommented:
What scripting language are you using? powershell, vbscript, perl, batch, etc.
Each one has their own methods to traverse directories, and looking at filenames and then performing a pattern match to see whether they meet the requirements or have invalid characters.
 
You need to provide what options are available to you with which you are familiar at which point ....... it would be simpler to guide you along to the solution.
0
LightWireAuthor Commented:
I'm not a script writer per say but can modify one when needed.  I am familiar with PS, VBS and batch
0
arnoldCommented:
Here is an example of listing files in directories and sub folders.
vbscript example and powershell example.

http://blogs.technet.com/b/heyscriptingguy/archive/2014/02/03/list-files-in-folders-and-subfolders-with-powershell.aspx

Are you looking for someone to write you the script?
or are you looking for suggestions/correction of your own script that is doing somethings but not all that you need?

What is the end goal of your script processing?
Are you looking to rename the filenames to meet the parameters of onedrive?

An example of a powershell script that will replace "inavalid" characters with an alternate valid one.
http://pshscripts.blogspot.com/2013/11/fix-filenameps1.html

The general premise is
list the current directory, if it is a file, check the naming convention.
If it is a directory, check the naming convention and then look further within.

You can do it iteratively.
Function to look through the directory that is being passed to it. and if it hits a directory calls the same function with the new path as the argument
find_files (c:\)
find_files(c:\somedirectory)
etc.
0
David Johnson, CD, MVPOwnerCommented:
from a previous solution
if this is your first time running powershell .. start powershell, pin it to your taskbar, exit powershell, now on your taskbar right click the powershell icon, select run as administrator
once started
"set-executionpolicy remotesigned" (no quotes) press y to accept the policy change
cd to the directory you saved this script
type check-ill then press tab for tab completion you will see
.\check-illegalcharacters.ps1
you can add a space and then press tab to enter the required variables or just run the script and it will ask you to fill in some values..

if you need help type "help check-illegalcharacters.ps1 -examples
checkillegalcharacters.ps1 -path c:\temp -fix
This will attempt to fix invalid filenames but cannot fix path's > 127 characters you will have to fix this yourself
 
check-illegalcharacters.ps1
  <#
            .AUTHORS
            RezSupport
            David Johnson (ve3ofa)

            .VERSION
            1.01

            .FILENAME
          Save as "Check-IllegalCharacters.ps1"
           
           .SYNOPSIS 
          Checks and Optionally Fixes Files with Illegal Characters
          or FilePaths > 127 Characters

           .DESCRIPTION
          The Check-IllegalCharacters.ps1 script checks the path given
           for files with illegal characters and filelenghts greater than 
           127 characters for input to Onedrive
              
           .PARAMETER Path
           Specifies the path to the Root Directory to Search

           .PARAMETER -Fix
           Will Fix files with Illegal Filenames

           .PARAMETER -Verbose
           Will Give Verbose Output 
           
           .INPUTS
           None. You cannot pipe objects to Check-IllegalCharacters.ps1.

           .OUTPUTS
           None. Check-IllegalCharacters.ps1 does not generate any output.

           .EXAMPLE
           C:\PS> .\Check-IllegalCharacters.ps1 -path c:\

           .EXAMPLE
           C:\PS> .\Check-IllegalCharacters.ps1 -path C:\Data\ -Fix

           .EXAMPLE
           C:\PS> .\Check-IllegalCharacters.ps1 -path C:\Data\ - Verbose
           #>


[CmdletBinding()]
Param(
    [Parameter(Mandatory=$True)]
        [string]$Path,
        [Parameter(Mandatory=$False)]
        [switch] $Fix
        )       
    if (test-path $path) {
    Write-Host Checking files in $Path, please wait...
    #Get all files and folders under the path specified
    $items = Get-ChildItem -Path $Path -Recurse
    foreach ($item in $items)
    {
        #Check if the item is a file or a folder
        if ($item.PSIsContainer) { $type = "Folder" }
        else { $type = "File" }
       
        #Report item has been found if verbose mode is selected
        if ($Verbose) { Write-Host Found a $type called $item.FullName }
       
        #Check if item name is 128 characters or more in length
        if ($item.Name.Length -gt 127)
        {
            Write-Host $type $item.Name is 128 characters or over and will need to be truncated -ForegroundColor Red
        }
        else
        {
            #Got this from http://powershell.com/cs/blogs/tips/archive/2011/05/20/finding-multiple-regex-matches.aspx
            $illegalChars = '[&{}~#%]'
            filter Matches($illegalChars)
            {
                $item.Name | Select-String -AllMatches $illegalChars |
                Select-Object -ExpandProperty Matches
                Select-Object -ExpandProperty Values
            }
           
            #Replace illegal characters with legal characters where found
            $newFileName = $item.Name
            Matches $illegalChars | ForEach-Object {
                Write-Host $type $item.FullName has the illegal character $_.Value -ForegroundColor Red
                #These characters may be used on the file system but not SharePoint
                if ($_.Value -match "&") { $newFileName = ($newFileName -replace "&", "and") }
                if ($_.Value -match "{") { $newFileName = ($newFileName -replace "{", "(") }
                if ($_.Value -match "}") { $newFileName = ($newFileName -replace "}", ")") }
                if ($_.Value -match "~") { $newFileName = ($newFileName -replace "~", "-") }
                if ($_.Value -match "#") { $newFileName = ($newFileName -replace "#", "") }
                if ($_.Value -match "%") { $newFileName = ($newFileName -replace "%", "") }
            }
           
            #Check for start, end and double periods
            if ($newFileName.StartsWith(".")) { Write-Host $type $item.FullName starts with a period -ForegroundColor red }
            while ($newFileName.StartsWith(".")) { $newFileName = $newFileName.TrimStart(".") }
            if ($newFileName.EndsWith(".")) { Write-Host $type $item.FullName ends with a period -ForegroundColor Red }
            while ($newFileName.EndsWith("."))   { $newFileName = $newFileName.TrimEnd(".") }
            if ($newFileName.Contains("..")) { Write-Host $type $item.FullName contains double periods -ForegroundColor red }
            while ($newFileName.Contains(".."))  { $newFileName = $newFileName.Replace("..", ".") }
           
            #Fix file and folder names if found and the Fix switch is specified
            if (($newFileName -ne $item.Name) -and ($Fix))
            {
                Rename-Item $item.FullName -NewName ($newFileName)
                Write-Host $type $item.Name has been changed to $newFileName -ForegroundColor Blue
            }
        }
    }
 }
 else {
 Write-host("Path " + $path + " Does Not Exist")
 }
# SIG # Begin signature block
# MIINGAYJKoZIhvcNAQcCoIINCTCCDQUCAQExCzAJBgUrDgMCGgUAMGkGCisGAQQB
# gjcCAQSgWzBZMDQGCisGAQQBgjcCAR4wJgIDAQAABBAfzDtgWUsITrck0sYpfvNR
# AgEAAgEAAgEAAgEAAgEAMCEwCQYFKw4DAhoFAAQUP4Chs1Fb2XxmcAyA6UEi8rvt
# /xGgggpaMIIFIjCCBAqgAwIBAgIQAupQIxjzGlMFoE+9rHncOTANBgkqhkiG9w0B
# AQsFADByMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYD
# VQQLExB3d3cuZGlnaWNlcnQuY29tMTEwLwYDVQQDEyhEaWdpQ2VydCBTSEEyIEFz
# c3VyZWQgSUQgQ29kZSBTaWduaW5nIENBMB4XDTE0MDcxNzAwMDAwMFoXDTE1MDcy
# MjEyMDAwMFowaTELMAkGA1UEBhMCQ0ExCzAJBgNVBAgTAk9OMREwDwYDVQQHEwhI
# YW1pbHRvbjEcMBoGA1UEChMTRGF2aWQgV2F5bmUgSm9obnNvbjEcMBoGA1UEAxMT
# RGF2aWQgV2F5bmUgSm9obnNvbjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
# ggEBAM3+T+61MoGxUHnoK0b2GgO17e0sW8ugwAH966Z1JIzQvXFa707SZvTJgmra
# ZsCn9fU+i9KhC0nUpA4hAv/b1MCeqGq1O0f3ffiwsxhTG3Z4J8mEl5eSdcRgeb+1
# jaKI3oHkbX+zxqOLSaRSQPn3XygMAfrcD/QI4vsx8o2lTUsPJEy2c0z57e1VzWlq
# KHqo18lVxDq/YF+fKCAJL57zjXSBPPmb/sNj8VgoxXS6EUAC5c3tb+CJfNP2U9vV
# oy5YeUP9bNwq2aXkW0+xZIipbJonZwN+bIsbgCC5eb2aqapBgJrgds8cw8WKiZvy
# Zx2qT7hy9HT+LUOI0l0K0w31dF8CAwEAAaOCAbswggG3MB8GA1UdIwQYMBaAFFrE
# uXsqCqOl6nEDwGD5LfZldQ5YMB0GA1UdDgQWBBTnMIKoGnZIswBx8nuJckJGsFDU
# lDAOBgNVHQ8BAf8EBAMCB4AwEwYDVR0lBAwwCgYIKwYBBQUHAwMwdwYDVR0fBHAw
# bjA1oDOgMYYvaHR0cDovL2NybDMuZGlnaWNlcnQuY29tL3NoYTItYXNzdXJlZC1j
# cy1nMS5jcmwwNaAzoDGGL2h0dHA6Ly9jcmw0LmRpZ2ljZXJ0LmNvbS9zaGEyLWFz
# c3VyZWQtY3MtZzEuY3JsMEIGA1UdIAQ7MDkwNwYJYIZIAYb9bAMBMCowKAYIKwYB
# BQUHAgEWHGh0dHBzOi8vd3d3LmRpZ2ljZXJ0LmNvbS9DUFMwgYQGCCsGAQUFBwEB
# BHgwdjAkBggrBgEFBQcwAYYYaHR0cDovL29jc3AuZGlnaWNlcnQuY29tME4GCCsG
# AQUFBzAChkJodHRwOi8vY2FjZXJ0cy5kaWdpY2VydC5jb20vRGlnaUNlcnRTSEEy
# QXNzdXJlZElEQ29kZVNpZ25pbmdDQS5jcnQwDAYDVR0TAQH/BAIwADANBgkqhkiG
# 9w0BAQsFAAOCAQEAVlkBmOEKRw2O66aloy9tNoQNIWz3AduGBfnf9gvyRFvSuKm0
# Zq3A6lRej8FPxC5Kbwswxtl2L/pjyrlYzUs+XuYe9Ua9YMIdhbyjUol4Z46jhOrO
# TDl18txaoNpGE9JXo8SLZHibwz97H3+paRm16aygM5R3uQ0xSQ1NFqDJ53YRvOqT
# 60/tF9E8zNx4hOH1lw1CDPu0K3nL2PusLUVzCpwNunQzGoZfVtlnV2x4EgXyZ9G1
# x4odcYZwKpkWPKA4bWAG+Img5+dgGEOqoUHh4jm2IKijm1jz7BRcJUMAwa2Qcbc2
# ttQbSj/7xZXL470VG3WjLWNWkRaRQAkzOajhpTCCBTAwggQYoAMCAQICEAQJGBtf
# 1btmdVNDtW+VUAgwDQYJKoZIhvcNAQELBQAwZTELMAkGA1UEBhMCVVMxFTATBgNV
# BAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3LmRpZ2ljZXJ0LmNvbTEkMCIG
# A1UEAxMbRGlnaUNlcnQgQXNzdXJlZCBJRCBSb290IENBMB4XDTEzMTAyMjEyMDAw
# MFoXDTI4MTAyMjEyMDAwMFowcjELMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lD
# ZXJ0IEluYzEZMBcGA1UECxMQd3d3LmRpZ2ljZXJ0LmNvbTExMC8GA1UEAxMoRGln
# aUNlcnQgU0hBMiBBc3N1cmVkIElEIENvZGUgU2lnbmluZyBDQTCCASIwDQYJKoZI
# hvcNAQEBBQADggEPADCCAQoCggEBAPjTsxx/DhGvZ3cH0wsxSRnP0PtFmbE620T1
# f+Wondsy13Hqdp0FLreP+pJDwKX5idQ3Gde2qvCchqXYJawOeSg6funRZ9PG+ykn
# x9N7I5TkkSOWkHeC+aGEI2YSVDNQdLEoJrskacLCUvIUZ4qJRdQtoaPpiCwgla4c
# SocI3wz14k1gGL6qxLKucDFmM3E+rHCiq85/6XzLkqHlOzEcz+ryCuRXu0q16XTm
# K/5sy350OTYNkO/ktU6kqepqCquE86xnTrXE94zRICUj6whkPlKWwfIPEvTFjg/B
# ougsUfdzvL2FsWKDc0GCB+Q4i2pzINAPZHM8np+mM6n9Gd8lk9ECAwEAAaOCAc0w
# ggHJMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMBMGA1UdJQQM
# MAoGCCsGAQUFBwMDMHkGCCsGAQUFBwEBBG0wazAkBggrBgEFBQcwAYYYaHR0cDov
# L29jc3AuZGlnaWNlcnQuY29tMEMGCCsGAQUFBzAChjdodHRwOi8vY2FjZXJ0cy5k
# aWdpY2VydC5jb20vRGlnaUNlcnRBc3N1cmVkSURSb290Q0EuY3J0MIGBBgNVHR8E
# ejB4MDqgOKA2hjRodHRwOi8vY3JsNC5kaWdpY2VydC5jb20vRGlnaUNlcnRBc3N1
# cmVkSURSb290Q0EuY3JsMDqgOKA2hjRodHRwOi8vY3JsMy5kaWdpY2VydC5jb20v
# RGlnaUNlcnRBc3N1cmVkSURSb290Q0EuY3JsME8GA1UdIARIMEYwOAYKYIZIAYb9
# bAACBDAqMCgGCCsGAQUFBwIBFhxodHRwczovL3d3dy5kaWdpY2VydC5jb20vQ1BT
# MAoGCGCGSAGG/WwDMB0GA1UdDgQWBBRaxLl7KgqjpepxA8Bg+S32ZXUOWDAfBgNV
# HSMEGDAWgBRF66Kv9JLLgjEtUYunpyGd823IDzANBgkqhkiG9w0BAQsFAAOCAQEA
# PuwNWiSz8yLRFcgsfCUpdqgdXRwtOhrE7zBh134LYP3DPQ/Er4v97yrfIFU3sOH2
# 0ZJ1D1G0bqWOWuJeJIFOEKTuP3GOYw4TS63XX0R58zYUBor3nEZOXP+QsRsHDpEV
# +7qvtVHCjSSuJMbHJyqhKSgaOnEoAjwukaPAJRHinBRHoXpoaK+bp1wgXNlxsQyP
# u6j4xRJon89Ay0BEpRPw5mQMJQhCMrI2iiQC/i9yfhzXSUWW6Fkd6fp0ZGuy62ZD
# 2rOwjNXpDd32ASDOmTFjPQgaGLOBm0/GkxAG/AeB+ova+YJJ92JuoVP6EpQYhS6S
# kepobEQysmah5xikmmRR7zGCAigwggIkAgEBMIGGMHIxCzAJBgNVBAYTAlVTMRUw
# EwYDVQQKEwxEaWdpQ2VydCBJbmMxGTAXBgNVBAsTEHd3dy5kaWdpY2VydC5jb20x
# MTAvBgNVBAMTKERpZ2lDZXJ0IFNIQTIgQXNzdXJlZCBJRCBDb2RlIFNpZ25pbmcg
# Q0ECEALqUCMY8xpTBaBPvax53DkwCQYFKw4DAhoFAKB4MBgGCisGAQQBgjcCAQwx
# CjAIoAKAAKECgAAwGQYJKoZIhvcNAQkDMQwGCisGAQQBgjcCAQQwHAYKKwYBBAGC
# NwIBCzEOMAwGCisGAQQBgjcCARUwIwYJKoZIhvcNAQkEMRYEFGgbs9EWp7NO+aFG
# mhxtwHAWf+WAMA0GCSqGSIb3DQEBAQUABIIBALe0kGB/GPbwfsvuUToffqfkIazu
# 1bKzXnknhWlYGrmrubhRYAWbmoIh6xnQP16aNYVg/dMk5RuWwg2+jV9w4mGf3GGz
# ztjHKH9A8/DzWjlAonSAXZ6EwmFGROf68BbgtYdDcdp0kAVHu2rYihS+Ia2os9r1
# MJ9m/ui/l/pUwXDQbhZirDh1wWk3cVOjZHqfS3C0w2ORaYRTiu0vawD/2u3jlt7N
# KlbyYwzUEB0VVqjIfGMdRTitm/PELYsfGDaI/nx5eUYv5jTqmFBqNFzeeBCHJFuW
# r5yS96GymJwXh3kVL0iDnLCQNiFzaBMDE/J5lp4OBEFj7++tVjnTFi4ZWrc=
# SIG # End signature block

Open in new window

0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
LightWireAuthor Commented:
I seen a ps script like this one as well which I will run tomorrow. Thx
0
LightWireAuthor Commented:
I've requested that this question be closed as follows:

Accepted answer: 0 points for lightwireinc's comment #a40686146

for the following reason:

Script runs good and helped find what I needed
0
arnoldCommented:
I think you forgot to choose the comment with the script that helped you resolve the issue at hand. Your own comment acknowledges the assistance and is not a solution.
0
LightWireAuthor Commented:
This script was very helpful and was what I was trying to use from another find but did not work properly.  I did modify the search by adding the following characters


            $illegalChars = '[&{}~#%/:*?<>|]'

#These characters may be used on the file system but not SharePoint
                if ($_.Value -match "&") { $newFileName = ($newFileName -replace "&", "and") }
                if ($_.Value -match "{") { $newFileName = ($newFileName -replace "{", "(") }
                if ($_.Value -match "}") { $newFileName = ($newFileName -replace "}", ")") }
                if ($_.Value -match "~") { $newFileName = ($newFileName -replace "~", "-") }
                if ($_.Value -match "#") { $newFileName = ($newFileName -replace "#", "") }
                if ($_.Value -match "%") { $newFileName = ($newFileName -replace "%", "") }
            if ($_.Value -match "/") { $newFileName = ($newFileName -replace "/", "") }
            if ($_.Value -match ":") { $newFileName = ($newFileName -replace ":", "") }
            if ($_.Value -match "'*") { $newFileName = ($newFileName -replace "'*", "") }
            if ($_.Value -match "'?") { $newFileName = ($newFileName -replace "'?", "") }
              if ($_.Value -match "<") { $newFileName = ($newFileName -replace "<", "") }
            if ($_.Value -match ">") { $newFileName = ($newFileName -replace ">", "") }
            if ($_.Value -match "|") { $newFileName = ($newFileName -replace "|", "") }
            }
0
LightWireAuthor Commented:
Sorry arnold, I chose the wrong thing.
0
howard brooksCommented:
The Long path tool is the very best program for error, unlock solution.and it is very useful tool.
0
Anand Shaw ShawCommented:
“Long Path Tool” is very helpful for this error ! best solution for your problem.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Office 365

From novice to tech pro — start learning today.