• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 2777
  • Last Modified:

How to find or search for invalid characters for OneDrive upload

Looking how to find or search for the following invalid characters in all the shared folders to prepare to upload to OneDrive.  

The characters are:

\
/
:
*
?



|
#
%
0
LightWire
Asked:
LightWire
1 Solution
 
arnoldCommented:
What are you using to upload? a pattern match in your application to validate the file name .......
0
 
LightWireAuthor Commented:
I need to verify my data structure before I proceed in the upload.  Is there a script I can use to produce an list of the output so it can be modified?
0
 
arnoldCommented:
Yes.

Depending on what it is you are using, you have to look at the naming convention you use and make sure it matches the requirements .
0
Cloud Class® Course: MCSA MCSE Windows Server 2012

This course teaches how to install and configure Windows Server 2012 R2.  It is the first step on your path to becoming a Microsoft Certified Solutions Expert (MCSE).

 
LightWireAuthor Commented:
I understand, but this is a customer's structure and they have over 100k + files/folders and I would like to search via a script instead of one at a time.
0
 
arnoldCommented:
What scripting language are you using? powershell, vbscript, perl, batch, etc.
Each one has their own methods to traverse directories, and looking at filenames and then performing a pattern match to see whether they meet the requirements or have invalid characters.
 
You need to provide what options are available to you with which you are familiar at which point ....... it would be simpler to guide you along to the solution.
0
 
LightWireAuthor Commented:
I'm not a script writer per say but can modify one when needed.  I am familiar with PS, VBS and batch
0
 
arnoldCommented:
Here is an example of listing files in directories and sub folders.
vbscript example and powershell example.

http://blogs.technet.com/b/heyscriptingguy/archive/2014/02/03/list-files-in-folders-and-subfolders-with-powershell.aspx

Are you looking for someone to write you the script?
or are you looking for suggestions/correction of your own script that is doing somethings but not all that you need?

What is the end goal of your script processing?
Are you looking to rename the filenames to meet the parameters of onedrive?

An example of a powershell script that will replace "inavalid" characters with an alternate valid one.
http://pshscripts.blogspot.com/2013/11/fix-filenameps1.html

The general premise is
list the current directory, if it is a file, check the naming convention.
If it is a directory, check the naming convention and then look further within.

You can do it iteratively.
Function to look through the directory that is being passed to it. and if it hits a directory calls the same function with the new path as the argument
find_files (c:\)
find_files(c:\somedirectory)
etc.
0
 
David Johnson, CD, MVPOwnerCommented:
from a previous solution
if this is your first time running powershell .. start powershell, pin it to your taskbar, exit powershell, now on your taskbar right click the powershell icon, select run as administrator
once started
"set-executionpolicy remotesigned" (no quotes) press y to accept the policy change
cd to the directory you saved this script
type check-ill then press tab for tab completion you will see
.\check-illegalcharacters.ps1
you can add a space and then press tab to enter the required variables or just run the script and it will ask you to fill in some values..

if you need help type "help check-illegalcharacters.ps1 -examples
checkillegalcharacters.ps1 -path c:\temp -fix
This will attempt to fix invalid filenames but cannot fix path's > 127 characters you will have to fix this yourself
 
check-illegalcharacters.ps1
  <#
            .AUTHORS
            RezSupport
            David Johnson (ve3ofa)

            .VERSION
            1.01

            .FILENAME
          Save as "Check-IllegalCharacters.ps1"
           
           .SYNOPSIS 
          Checks and Optionally Fixes Files with Illegal Characters
          or FilePaths > 127 Characters

           .DESCRIPTION
          The Check-IllegalCharacters.ps1 script checks the path given
           for files with illegal characters and filelenghts greater than 
           127 characters for input to Onedrive
              
           .PARAMETER Path
           Specifies the path to the Root Directory to Search

           .PARAMETER -Fix
           Will Fix files with Illegal Filenames

           .PARAMETER -Verbose
           Will Give Verbose Output 
           
           .INPUTS
           None. You cannot pipe objects to Check-IllegalCharacters.ps1.

           .OUTPUTS
           None. Check-IllegalCharacters.ps1 does not generate any output.

           .EXAMPLE
           C:\PS> .\Check-IllegalCharacters.ps1 -path c:\

           .EXAMPLE
           C:\PS> .\Check-IllegalCharacters.ps1 -path C:\Data\ -Fix

           .EXAMPLE
           C:\PS> .\Check-IllegalCharacters.ps1 -path C:\Data\ - Verbose
           #>


[CmdletBinding()]
Param(
    [Parameter(Mandatory=$True)]
        [string]$Path,
        [Parameter(Mandatory=$False)]
        [switch] $Fix
        )       
    if (test-path $path) {
    Write-Host Checking files in $Path, please wait...
    #Get all files and folders under the path specified
    $items = Get-ChildItem -Path $Path -Recurse
    foreach ($item in $items)
    {
        #Check if the item is a file or a folder
        if ($item.PSIsContainer) { $type = "Folder" }
        else { $type = "File" }
       
        #Report item has been found if verbose mode is selected
        if ($Verbose) { Write-Host Found a $type called $item.FullName }
       
        #Check if item name is 128 characters or more in length
        if ($item.Name.Length -gt 127)
        {
            Write-Host $type $item.Name is 128 characters or over and will need to be truncated -ForegroundColor Red
        }
        else
        {
            #Got this from http://powershell.com/cs/blogs/tips/archive/2011/05/20/finding-multiple-regex-matches.aspx
            $illegalChars = '[&{}~#%]'
            filter Matches($illegalChars)
            {
                $item.Name | Select-String -AllMatches $illegalChars |
                Select-Object -ExpandProperty Matches
                Select-Object -ExpandProperty Values
            }
           
            #Replace illegal characters with legal characters where found
            $newFileName = $item.Name
            Matches $illegalChars | ForEach-Object {
                Write-Host $type $item.FullName has the illegal character $_.Value -ForegroundColor Red
                #These characters may be used on the file system but not SharePoint
                if ($_.Value -match "&") { $newFileName = ($newFileName -replace "&", "and") }
                if ($_.Value -match "{") { $newFileName = ($newFileName -replace "{", "(") }
                if ($_.Value -match "}") { $newFileName = ($newFileName -replace "}", ")") }
                if ($_.Value -match "~") { $newFileName = ($newFileName -replace "~", "-") }
                if ($_.Value -match "#") { $newFileName = ($newFileName -replace "#", "") }
                if ($_.Value -match "%") { $newFileName = ($newFileName -replace "%", "") }
            }
           
            #Check for start, end and double periods
            if ($newFileName.StartsWith(".")) { Write-Host $type $item.FullName starts with a period -ForegroundColor red }
            while ($newFileName.StartsWith(".")) { $newFileName = $newFileName.TrimStart(".") }
            if ($newFileName.EndsWith(".")) { Write-Host $type $item.FullName ends with a period -ForegroundColor Red }
            while ($newFileName.EndsWith("."))   { $newFileName = $newFileName.TrimEnd(".") }
            if ($newFileName.Contains("..")) { Write-Host $type $item.FullName contains double periods -ForegroundColor red }
            while ($newFileName.Contains(".."))  { $newFileName = $newFileName.Replace("..", ".") }
           
            #Fix file and folder names if found and the Fix switch is specified
            if (($newFileName -ne $item.Name) -and ($Fix))
            {
                Rename-Item $item.FullName -NewName ($newFileName)
                Write-Host $type $item.Name has been changed to $newFileName -ForegroundColor Blue
            }
        }
    }
 }
 else {
 Write-host("Path " + $path + " Does Not Exist")
 }
# SIG # Begin signature block
# MIINGAYJKoZIhvcNAQcCoIINCTCCDQUCAQExCzAJBgUrDgMCGgUAMGkGCisGAQQB
# gjcCAQSgWzBZMDQGCisGAQQBgjcCAR4wJgIDAQAABBAfzDtgWUsITrck0sYpfvNR
# AgEAAgEAAgEAAgEAAgEAMCEwCQYFKw4DAhoFAAQUP4Chs1Fb2XxmcAyA6UEi8rvt
# /xGgggpaMIIFIjCCBAqgAwIBAgIQAupQIxjzGlMFoE+9rHncOTANBgkqhkiG9w0B
# AQsFADByMQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYD
# VQQLExB3d3cuZGlnaWNlcnQuY29tMTEwLwYDVQQDEyhEaWdpQ2VydCBTSEEyIEFz
# c3VyZWQgSUQgQ29kZSBTaWduaW5nIENBMB4XDTE0MDcxNzAwMDAwMFoXDTE1MDcy
# MjEyMDAwMFowaTELMAkGA1UEBhMCQ0ExCzAJBgNVBAgTAk9OMREwDwYDVQQHEwhI
# YW1pbHRvbjEcMBoGA1UEChMTRGF2aWQgV2F5bmUgSm9obnNvbjEcMBoGA1UEAxMT
# RGF2aWQgV2F5bmUgSm9obnNvbjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
# ggEBAM3+T+61MoGxUHnoK0b2GgO17e0sW8ugwAH966Z1JIzQvXFa707SZvTJgmra
# ZsCn9fU+i9KhC0nUpA4hAv/b1MCeqGq1O0f3ffiwsxhTG3Z4J8mEl5eSdcRgeb+1
# jaKI3oHkbX+zxqOLSaRSQPn3XygMAfrcD/QI4vsx8o2lTUsPJEy2c0z57e1VzWlq
# KHqo18lVxDq/YF+fKCAJL57zjXSBPPmb/sNj8VgoxXS6EUAC5c3tb+CJfNP2U9vV
# oy5YeUP9bNwq2aXkW0+xZIipbJonZwN+bIsbgCC5eb2aqapBgJrgds8cw8WKiZvy
# Zx2qT7hy9HT+LUOI0l0K0w31dF8CAwEAAaOCAbswggG3MB8GA1UdIwQYMBaAFFrE
# uXsqCqOl6nEDwGD5LfZldQ5YMB0GA1UdDgQWBBTnMIKoGnZIswBx8nuJckJGsFDU
# lDAOBgNVHQ8BAf8EBAMCB4AwEwYDVR0lBAwwCgYIKwYBBQUHAwMwdwYDVR0fBHAw
# bjA1oDOgMYYvaHR0cDovL2NybDMuZGlnaWNlcnQuY29tL3NoYTItYXNzdXJlZC1j
# cy1nMS5jcmwwNaAzoDGGL2h0dHA6Ly9jcmw0LmRpZ2ljZXJ0LmNvbS9zaGEyLWFz
# c3VyZWQtY3MtZzEuY3JsMEIGA1UdIAQ7MDkwNwYJYIZIAYb9bAMBMCowKAYIKwYB
# BQUHAgEWHGh0dHBzOi8vd3d3LmRpZ2ljZXJ0LmNvbS9DUFMwgYQGCCsGAQUFBwEB
# BHgwdjAkBggrBgEFBQcwAYYYaHR0cDovL29jc3AuZGlnaWNlcnQuY29tME4GCCsG
# AQUFBzAChkJodHRwOi8vY2FjZXJ0cy5kaWdpY2VydC5jb20vRGlnaUNlcnRTSEEy
# QXNzdXJlZElEQ29kZVNpZ25pbmdDQS5jcnQwDAYDVR0TAQH/BAIwADANBgkqhkiG
# 9w0BAQsFAAOCAQEAVlkBmOEKRw2O66aloy9tNoQNIWz3AduGBfnf9gvyRFvSuKm0
# Zq3A6lRej8FPxC5Kbwswxtl2L/pjyrlYzUs+XuYe9Ua9YMIdhbyjUol4Z46jhOrO
# TDl18txaoNpGE9JXo8SLZHibwz97H3+paRm16aygM5R3uQ0xSQ1NFqDJ53YRvOqT
# 60/tF9E8zNx4hOH1lw1CDPu0K3nL2PusLUVzCpwNunQzGoZfVtlnV2x4EgXyZ9G1
# x4odcYZwKpkWPKA4bWAG+Img5+dgGEOqoUHh4jm2IKijm1jz7BRcJUMAwa2Qcbc2
# ttQbSj/7xZXL470VG3WjLWNWkRaRQAkzOajhpTCCBTAwggQYoAMCAQICEAQJGBtf
# 1btmdVNDtW+VUAgwDQYJKoZIhvcNAQELBQAwZTELMAkGA1UEBhMCVVMxFTATBgNV
# BAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3LmRpZ2ljZXJ0LmNvbTEkMCIG
# A1UEAxMbRGlnaUNlcnQgQXNzdXJlZCBJRCBSb290IENBMB4XDTEzMTAyMjEyMDAw
# MFoXDTI4MTAyMjEyMDAwMFowcjELMAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lD
# ZXJ0IEluYzEZMBcGA1UECxMQd3d3LmRpZ2ljZXJ0LmNvbTExMC8GA1UEAxMoRGln
# aUNlcnQgU0hBMiBBc3N1cmVkIElEIENvZGUgU2lnbmluZyBDQTCCASIwDQYJKoZI
# hvcNAQEBBQADggEPADCCAQoCggEBAPjTsxx/DhGvZ3cH0wsxSRnP0PtFmbE620T1
# f+Wondsy13Hqdp0FLreP+pJDwKX5idQ3Gde2qvCchqXYJawOeSg6funRZ9PG+ykn
# x9N7I5TkkSOWkHeC+aGEI2YSVDNQdLEoJrskacLCUvIUZ4qJRdQtoaPpiCwgla4c
# SocI3wz14k1gGL6qxLKucDFmM3E+rHCiq85/6XzLkqHlOzEcz+ryCuRXu0q16XTm
# K/5sy350OTYNkO/ktU6kqepqCquE86xnTrXE94zRICUj6whkPlKWwfIPEvTFjg/B
# ougsUfdzvL2FsWKDc0GCB+Q4i2pzINAPZHM8np+mM6n9Gd8lk9ECAwEAAaOCAc0w
# ggHJMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMBMGA1UdJQQM
# MAoGCCsGAQUFBwMDMHkGCCsGAQUFBwEBBG0wazAkBggrBgEFBQcwAYYYaHR0cDov
# L29jc3AuZGlnaWNlcnQuY29tMEMGCCsGAQUFBzAChjdodHRwOi8vY2FjZXJ0cy5k
# aWdpY2VydC5jb20vRGlnaUNlcnRBc3N1cmVkSURSb290Q0EuY3J0MIGBBgNVHR8E
# ejB4MDqgOKA2hjRodHRwOi8vY3JsNC5kaWdpY2VydC5jb20vRGlnaUNlcnRBc3N1
# cmVkSURSb290Q0EuY3JsMDqgOKA2hjRodHRwOi8vY3JsMy5kaWdpY2VydC5jb20v
# RGlnaUNlcnRBc3N1cmVkSURSb290Q0EuY3JsME8GA1UdIARIMEYwOAYKYIZIAYb9
# bAACBDAqMCgGCCsGAQUFBwIBFhxodHRwczovL3d3dy5kaWdpY2VydC5jb20vQ1BT
# MAoGCGCGSAGG/WwDMB0GA1UdDgQWBBRaxLl7KgqjpepxA8Bg+S32ZXUOWDAfBgNV
# HSMEGDAWgBRF66Kv9JLLgjEtUYunpyGd823IDzANBgkqhkiG9w0BAQsFAAOCAQEA
# PuwNWiSz8yLRFcgsfCUpdqgdXRwtOhrE7zBh134LYP3DPQ/Er4v97yrfIFU3sOH2
# 0ZJ1D1G0bqWOWuJeJIFOEKTuP3GOYw4TS63XX0R58zYUBor3nEZOXP+QsRsHDpEV
# +7qvtVHCjSSuJMbHJyqhKSgaOnEoAjwukaPAJRHinBRHoXpoaK+bp1wgXNlxsQyP
# u6j4xRJon89Ay0BEpRPw5mQMJQhCMrI2iiQC/i9yfhzXSUWW6Fkd6fp0ZGuy62ZD
# 2rOwjNXpDd32ASDOmTFjPQgaGLOBm0/GkxAG/AeB+ova+YJJ92JuoVP6EpQYhS6S
# kepobEQysmah5xikmmRR7zGCAigwggIkAgEBMIGGMHIxCzAJBgNVBAYTAlVTMRUw
# EwYDVQQKEwxEaWdpQ2VydCBJbmMxGTAXBgNVBAsTEHd3dy5kaWdpY2VydC5jb20x
# MTAvBgNVBAMTKERpZ2lDZXJ0IFNIQTIgQXNzdXJlZCBJRCBDb2RlIFNpZ25pbmcg
# Q0ECEALqUCMY8xpTBaBPvax53DkwCQYFKw4DAhoFAKB4MBgGCisGAQQBgjcCAQwx
# CjAIoAKAAKECgAAwGQYJKoZIhvcNAQkDMQwGCisGAQQBgjcCAQQwHAYKKwYBBAGC
# NwIBCzEOMAwGCisGAQQBgjcCARUwIwYJKoZIhvcNAQkEMRYEFGgbs9EWp7NO+aFG
# mhxtwHAWf+WAMA0GCSqGSIb3DQEBAQUABIIBALe0kGB/GPbwfsvuUToffqfkIazu
# 1bKzXnknhWlYGrmrubhRYAWbmoIh6xnQP16aNYVg/dMk5RuWwg2+jV9w4mGf3GGz
# ztjHKH9A8/DzWjlAonSAXZ6EwmFGROf68BbgtYdDcdp0kAVHu2rYihS+Ia2os9r1
# MJ9m/ui/l/pUwXDQbhZirDh1wWk3cVOjZHqfS3C0w2ORaYRTiu0vawD/2u3jlt7N
# KlbyYwzUEB0VVqjIfGMdRTitm/PELYsfGDaI/nx5eUYv5jTqmFBqNFzeeBCHJFuW
# r5yS96GymJwXh3kVL0iDnLCQNiFzaBMDE/J5lp4OBEFj7++tVjnTFi4ZWrc=
# SIG # End signature block

Open in new window

0
 
LightWireAuthor Commented:
I seen a ps script like this one as well which I will run tomorrow. Thx
0
 
LightWireAuthor Commented:
I've requested that this question be closed as follows:

Accepted answer: 0 points for lightwireinc's comment #a40686146

for the following reason:

Script runs good and helped find what I needed
0
 
arnoldCommented:
I think you forgot to choose the comment with the script that helped you resolve the issue at hand. Your own comment acknowledges the assistance and is not a solution.
0
 
LightWireAuthor Commented:
This script was very helpful and was what I was trying to use from another find but did not work properly.  I did modify the search by adding the following characters


            $illegalChars = '[&{}~#%/:*?<>|]'

#These characters may be used on the file system but not SharePoint
                if ($_.Value -match "&") { $newFileName = ($newFileName -replace "&", "and") }
                if ($_.Value -match "{") { $newFileName = ($newFileName -replace "{", "(") }
                if ($_.Value -match "}") { $newFileName = ($newFileName -replace "}", ")") }
                if ($_.Value -match "~") { $newFileName = ($newFileName -replace "~", "-") }
                if ($_.Value -match "#") { $newFileName = ($newFileName -replace "#", "") }
                if ($_.Value -match "%") { $newFileName = ($newFileName -replace "%", "") }
            if ($_.Value -match "/") { $newFileName = ($newFileName -replace "/", "") }
            if ($_.Value -match ":") { $newFileName = ($newFileName -replace ":", "") }
            if ($_.Value -match "'*") { $newFileName = ($newFileName -replace "'*", "") }
            if ($_.Value -match "'?") { $newFileName = ($newFileName -replace "'?", "") }
              if ($_.Value -match "<") { $newFileName = ($newFileName -replace "<", "") }
            if ($_.Value -match ">") { $newFileName = ($newFileName -replace ">", "") }
            if ($_.Value -match "|") { $newFileName = ($newFileName -replace "|", "") }
            }
0
 
LightWireAuthor Commented:
Sorry arnold, I chose the wrong thing.
0
 
howard brooksCommented:
The Long path tool is the very best program for error, unlock solution.and it is very useful tool.
0
 
Anand Shaw ShawCommented:
“Long Path Tool” is very helpful for this error ! best solution for your problem.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

Join & Write a Comment

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now